US20100205475A1 - Meta-data driven, service-oriented architecture (soa)-enabled, application independent interface gateway - Google Patents
Meta-data driven, service-oriented architecture (soa)-enabled, application independent interface gateway Download PDFInfo
- Publication number
- US20100205475A1 US20100205475A1 US12/636,441 US63644109A US2010205475A1 US 20100205475 A1 US20100205475 A1 US 20100205475A1 US 63644109 A US63644109 A US 63644109A US 2010205475 A1 US2010205475 A1 US 2010205475A1
- Authority
- US
- United States
- Prior art keywords
- interface
- data
- component
- user
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6227—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database where protection concerns the structure of data, e.g. records, types, queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
Definitions
- a large enterprise may include a variety of data stores (e.g., operations support, enterprise resource planning (ERP), data warehouse (DW), enterprise performance management (EPM) applications, etc.), devices, and/or systems.
- ERP enterprise resource planning
- DW data warehouse
- EPM enterprise performance management
- the large enterprise may have difficulty providing end-to-end integration (e.g., communication integration, process integration between applications, etc.) between these data stores and/or devices given the diversity of the infrastructure.
- end-to-end integration e.g., communication integration, process integration between applications, etc.
- communication integration e.g., an adapter framework
- process integration between specific applications e.g., an Application Integration Architecture (AIA) framework
- AIA Application Integration Architecture
- FIG. 1 is an exemplary overview of a service integration platform
- FIG. 2 is a diagram of an exemplary environment according to implementations described herein;
- FIG. 3 is a diagram of exemplary components of a service integration platform of FIG. 2 ;
- FIG. 4 is a diagram of exemplary components of an interface gateway of FIG. 3 ;
- FIG. 5 is a diagram of exemplary components of a device
- FIG. 6 is a diagram of exemplary functional components of an orchestrator of FIG. 4 ;
- FIG. 7 is a diagram of an exemplary portion of a metadata database of FIG. 4 ;
- FIG. 8 is a diagram of exemplary functional components of a portal of FIG. 3 ;
- FIG. 9 is a flow chart of an exemplary process for configuring an interface at the interface gateway of FIG. 3 ;
- FIGS. 10A-10E are diagrams of exemplary graphical user interfaces that may be provided in connection with the process described with respect to FIG. 9 ;
- FIG. 11 is a flow chart of an exemplary process that may be performed by an interface gateway when operating in a real-time mode
- FIG. 12 is a flow chart of an exemplary process that may be performed by an interface gateway when operating in a batch mode
- FIG. 13 is a flow chart of an exemplary process for monitoring the execution of an interface.
- FIG. 14 is a flow chart of an exemplary process for performing error correction.
- a service integration platform may provide the service integration based on a service-oriented architecture (SOA).
- SOA service-oriented architecture
- FIG. 1 is a diagram of an exemplary overview 100 of a service integration platform.
- data (denoted as data A) is to be transferred from the purchasing department of a business desires to the financial department of the business.
- the data may include any type of information, such as information that is needed by the financial department, information that a user in the purchasing department desires to send to a user in the financial department, and/or other types of information.
- data A is in a first format and that the financial department requires that the information in data A be provided in a different format, shown as data B.
- a service integration platform may be provided that acts as a generic interface that can be used to interface various kinds of systems, such as Enterprise Resource Planning (ERP) systems, Business Warehouse systems, Legacy systems, web services, Business-to-Business (B2B) systems, etc. and/or data sources, such as BizFrame, VFrame, flat files, spreadsheets, etc., and/or other types of systems and/or data sources that may exist in a heterogeneous system.
- ERP Enterprise Resource Planning
- B2B Business-to-Business
- data sources such as BizFrame, VFrame, flat files, spreadsheets, etc.
- the service integration platform may receive data in one type of format and from one type of system, convert the data to another type of format, and transfer the data to another type of system.
- the user in the purchasing department, the user in the financial department, or another user may configure an interface of the service integration platform to convert data from the purchasing department system to a desired format of the financial department system.
- the user in the purchasing department may transfer data A to the service integration platform.
- the service integration platform may identify the appropriate interface for converting the information from data A, convert the information to the appropriate format used by the financial department system (to create data B), and transfer data B to the financial department system.
- FIG. 2 is a diagram of an exemplary environment 200 according to implementations described herein. As illustrated, environment 200 may include a user device 210 , a scheduler 220 , and a service integration platform 230 that interconnect via a network 240 .
- User device 210 may include one or more devices that can transmit a request to process data to service integration platform 230 in a real-time mode.
- user device 210 may include one or more devices that allow a user to interact with service integration platform 230 .
- user device 210 may include a desktop computer, a laptop computer, a mainframe computer, a server, a handheld computing device (such as a cellular telephone, a personal digital assistant (PDA), etc.), and/or some other type of computational device.
- User device 210 may connect to network 240 via a wired and/or wireless connection.
- Scheduler 220 may include one or more devices that can transmit a request to process data to service integration platform 230 in a batch mode.
- scheduler 220 may include a desktop computer, a laptop computer, a mainframe computer, a server, and/or some other type of computational device.
- Scheduler 220 may connect to network 240 via a wired and/or wireless connection.
- Service integration platform 230 may include one or more devices that provide service integration for heterogeneous systems and/or data sources.
- Service integration platform 230 may include various interfaces that allow for transfer of data between different types of systems. In one implementation, each interface may have a one-to-one correspondence between a first system/data source and a second system/data source so as to provide data conversions between the first system/data source and the second system/data source.
- service integration platform 230 may include one or more devices that allow a user to interact with service integration platform 230 to, for example, configure an interface on service integration platform 230 , monitor the processing of data at service integration platform 230 , perform error correction with respect to data being processed at service integration platform 230 , and/or perform other functions.
- Service integration platform 230 may include, for example, one or more mainframe computers, servers, desktop computers, laptop computers, and/or some other type of computational device.
- service integration platform 230 may operate in a real-time mode and a batch mode.
- service integration platform 230 may receive a request to process a single payload of data from a user device, such as user device 210 , identify an interface for processing the single payload, and process the single payload using the interface.
- service integration platform 230 may receive a request to process a large payload from a scheduler, such as scheduler 220 , identify an interface for processing the large payload, and process the large payload using the interface. Additional details regarding the real-time and batch modes are provided below in connection with FIGS. 11 and 12 , respectively.
- Network 240 may include one or more networks of any type (wired and/or wireless).
- network 240 may include a local area network (LAN), a wide area network (WAN), a telephone network, such as a Public Switched Telephone Network (PSTN) or a cellular network, a satellite network, an intranet, the Internet, a data network, a private network, or a combination of networks.
- LAN local area network
- WAN wide area network
- PSTN Public Switched Telephone Network
- FIG. 2 illustrates exemplary components of environment 200
- environment 200 may include additional devices and/or networks, fewer devices and/or networks, different devices and/or networks, and/or differently arranged devices and/or networks than those illustrated in FIG. 2 .
- one or more devices in FIG. 2 may perform one or more tasks described as being performed by one or more other devices in FIG. 2 .
- FIG. 3 is a diagram of exemplary components of service integration platform 230 .
- service integration platform 230 may include an interface gateway 310 and a portal 320 .
- Interface gateway 310 may include one or more devices that act as the heart of service integration platform 230 .
- Interface gateway 310 may provide the interfaces for processing data in real-time and batch modes.
- Interface gateway 310 may also provide monitoring services and error correction services.
- Interface gateway 310 may include, for example, one or more mainframe computers, servers, desktop computers, laptop computers, mainframe computers, and/or some other type of computational device.
- Portal 320 may include one or more devices that allow a user to interact with interface gateway 310 .
- portal 320 may provide graphical user interfaces to a user device, such as user device 210 , that allow a user to configure an interface on interface gateway 310 , monitor the processing of data at interface gateway 310 , perform error correction with respect to data being processed at interface gateway 310 , and/or perform other functions.
- portal 320 may authenticate a user prior to providing the graphical user interfaces.
- the graphical user interfaces may be provided as web pages.
- Portal 320 may include, for example, one or more mainframe computers, servers, desktop computers, laptop computers, and/or some other type of computational device.
- FIG. 3 illustrates exemplary components of service integration platform 230
- service integration platform 230 may include additional components, fewer components, different components, and/or differently arranged components than those illustrated in FIG. 3 . Additionally, or alternatively, one component in FIG. 3 may perform one or more tasks described as being performed by another component in FIG. 3 .
- FIG. 4 is a diagram of exemplary components of interface gateway 310 .
- interface gateway 310 may include a security component 405 , an Integrated Specific Service (ISS) component 410 , an orchestrator component 415 , a service adapter component 420 , a services component 425 , a cache component 430 , an application database component 435 , a metadata database 440 , a transactional database 445 , and a real time monitor component 450 .
- ISS Integrated Specific Service
- Security component 405 may include one or more devices that authenticate a received request.
- the authentication may include authenticating the device from which the request is received and/or the user with which the request is associated.
- security component 405 may receive a real-time request from user device 210 and may, in response to receiving the request, authenticate user device 210 .
- the real-time request may include information identifying user device 210 and security component 405 may use that information to authenticate user device 210 .
- security component 405 may send a request for information to user device 210 and may use information returned from that request to authenticate user device 210 .
- Security component 405 may further receive a batch request from scheduler 220 and may, in response to receiving the request, authenticate scheduler 220 .
- the batch request may include information identifying scheduler 220 and security component 405 may use that information to authenticate scheduler 220 .
- security component 405 may send a request for information to scheduler 220 and may use information returned from that request to authenticate scheduler 220 .
- Security component 405 may transfer received requests to ISS component 410 in the real-time mode and/or to orchestrator component 415 in the batch mode.
- ISS component 410 may include one or more devices to convert received data, in the real-time mode, to a canonical structure.
- the canonical structure may include a canonical eXtensible Markup Language (XML) structure.
- XML canonical eXtensible Markup Language
- ISS component 410 may convert the incoming data (regardless of the format) to the canonical XML structure.
- IDoc Intermediate Document
- Other types of canonical structures can alternatively be used.
- ISS component 410 may transfer the data in the canonical structure to orchestrator 415 .
- ISS component 410 may include a data router and a group of translators, where each translator translates a different data format to the canonical structure.
- the data router may receive data, detect the format of the received data, identify, based on the detected format, the appropriate translator for translating the data, and route the data to the identified translator.
- the translator may then translate the received data to the canonical structure.
- users may use different naming conventions for naming the data, where the different naming conventions identify the format of the data.
- the data router may use the naming convention of incoming data to identify the appropriate translator to which to route the data.
- Orchestrator component 415 may include one or more devices that act as the main component of interface gateway 310 . Orchestrator component 415 may determine, based on data from metadata database 440 , the other components of interface gateway 310 that need to be invoked to complete an interface. In one implementation, orchestrator component 415 may be implemented using Business Process Execution Language (BPEL). Orchestrator 415 may operate in an asynchronous manner in the batch mode and a synchronous and/or asynchronous manner in the real-time mode. Further details regarding orchestrator component 415 are provided below with respect to FIG. 6 .
- BPEL Business Process Execution Language
- Service adapter component 420 may include one or more devices that act as a service implementation channel to access the services in services component 425 .
- service adapter component 420 may include a group of adapters that allow for different types of services to be invoked.
- the group of adapters may include one or more Oracle Data Integration (ODI) adapters, SAP adapters, database adapters, legacy adapters, File Transfer Protocol (FTP) adapters, and/or other types of adapters.
- ODI Oracle Data Integration
- SAP adapters SAP adapters
- database adapters database adapters
- legacy adapters legacy adapters
- File Transfer Protocol (FTP) adapters and/or other types of adapters.
- Service adapter component 420 may enable a service-oriented architecture to exist in interface gateway 310 , where new services may be added to services component 425 without affecting the rest of interface gateway 310 .
- Services component 425 may include one or more devices that provide services for processing data.
- the services may include process monitoring services, masking engine services, data services (e.g., that allow connectivity to databases), data error handling services, correction and recycling services, notification services, error handling services, Secure File Transfer Protocol (SFTP) services, and/or other types of services.
- the services may be implemented as ODI services, SAP services, Java Web Services, Unix shell script, and/or other types of services.
- Cache component 430 may include one or more devices that may store data.
- cache component 430 may cache data (e.g., in one or more reference tables) from application database 435 that may be needed by services component 425 . In this way, data may be available to services component 425 faster than if the data is retrieved from application database 435 .
- Application database 435 may include one or more devices that store data that may be accessed by services component 425 . As indicated above, cache component 430 may retrieve data, from application database 435 , which may be needed by services component 425 .
- Metadata database 440 may include one or more devices that store metadata that define the interfaces of interface gateway 310 .
- metadata database 440 may store the parameters of each interface with which interface gateway 310 is associated, along with information identifying the rules to be executed for each interface, the servers that need to be called for each interface, the services to be executed for each interface, and/or other information needed for executing an interface. While one metadata database is shown, it will be appreciated that metadata database 440 may consist of multiple databases stored locally at interface gateway 310 , or stored at one or more different and possibly remote locations. Additional details regarding metadata database 440 are provided below with respect to FIG. 7 .
- Transactional database 445 may include one or more devices that store information relating to the processing of data by interface gateway 310 .
- transactional database 445 may store information relating to errors that occurred while processing data by interface gateway 310 .
- transactional database 445 may maintain error tables that store data errors that were captured while processing the data in interface gateway 310 . While one transactional database is shown, it will be appreciated that transactional database 445 may consist of multiple databases stored locally at interface gateway 310 , or stored at one or more different and possibly remote locations.
- Real time monitor component 450 may include one or more devices that may monitor the operation of orchestrator component 415 and/or other components of interface gateway 310 .
- orchestrator component 415 may transmit, in real time, status information to real time monitor component 450 .
- Real time monitor component 450 may receive this status information and, for example, provide the status information to portal 320 to allow a user to track the latest status of the execution of data using the interfaces of interface gateway 310 .
- interface gateway 310 may include additional components, fewer components, different components, and/or differently arranged components than those illustrated in FIG. 4 . Additionally, or alternatively, one or more components in FIG. 4 may perform one or more tasks described as being performed by one or more other components in FIG. 4 .
- FIG. 5 is a diagram of exemplary components of a device 500 that may correspond to one or more of the devices or components depicted in FIGS. 2-4 .
- device 500 may correspond to user device 210 , scheduler 220 , one or more components of interface gateway 310 , and/or one or more components of portal 320 .
- device 500 may include a processing system 505 , a memory 510 , an input component 515 , an output component 520 , and a communication interface 525 .
- Processing system 505 may include one or more processors, microprocessors, and/or another type of component that may interpret and execute instructions.
- processing system 505 may include one or more application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc.
- ASICs application specific integrated circuits
- FPGAs field programmable gate arrays
- Memory 510 may include a random access memory (RAM), a dynamic random access memory (DRAM), a ferroelectric random access memory (FRAM), a read only memory (ROM), a programmable read only memory (PROM), a flash memory, and/or some other type of memory.
- Memory 510 may additionally, or alternatively, include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid state disk, etc.) or some other type of computer-readable medium, along with a corresponding drive.
- a computer-readable medium may correspond to, for example, a physical memory device or a logical memory device.
- a logical memory device may include memory space within a single physical memory device or spread across multiple physical memory devices residing on one or more devices.
- Input component 515 may permit an operator to input information into device 500 .
- input component 515 may include a keyboard, a button, a switch, a knob, a touchpad, a display, and/or some other type of input component.
- Output component 520 may permit device 500 to output information to the operator.
- output component 520 may include a display, light emitting diodes (LEDs), a speaker, and/or some other type of output component.
- LEDs light emitting diodes
- Communication interface 525 may permit device 500 to communicate with other devices, networks, and/or systems.
- communication interface 520 may include some type of wireless and/or wired interface.
- device 500 may perform certain operations in response to processing system 505 executing software instructions contained in a computer-readable medium, such as memory 510 .
- the software instructions may be read into memory 510 from another computer-readable medium or from another device via communication interface 525 .
- the software instructions contained in memory 510 may cause processing system 505 to perform processes described herein.
- hardwired circuitry may be used in place of or in combination with software instructions to implement processes described herein.
- implementations described herein are not limited to any specific combination of hardware circuitry and software.
- FIG. 5 illustrates exemplary components of device 500
- device 500 may include additional components, fewer components, different components, and/or differently arranged components than those illustrated in FIG. 5 . Additionally, or alternatively, one or more components in FIG. 5 may perform one or more tasks described as being performed by one or more other components in FIG. 5 .
- FIG. 6 is a diagram of exemplary functional components of orchestrator component 415 .
- orchestrator component 415 may include a master component 610 , a preprocess component 620 , an initialization component 630 , a preparation component 640 , a process component 650 , a completion component 660 , and a notification engine 670 .
- the functional components of orchestrator component 415 may be implemented using one or more components of device 500 of FIG. 5 .
- Master component 610 may include one or more functional components that may be invoked when interface gateway 310 is operating in the real-time mode or the batch mode. Master component 610 may receive information from ISS 410 (in the real-time mode) or security component 405 (in the batch mode), identify an interface, and invoke the appropriate one or ones of preprocessing component 620 , initialization component 630 , preparation component 640 , process component 650 , and completion component 660 for executing the identified interface. In this way, master component 610 may control the execution of an interface for processing received data. In one implementation, master component 610 may be implemented using BPEL.
- Preprocess component 620 may include one or more functional components that may be invoked when interface gateway 310 is operating in the batch mode. Preprocess component 620 may receive information identifying one or more files to be processed and the location of the one or more files. Preprocess component 620 may use the received information to retrieve the one or more files from the location and store the one or more files in a target location.
- Initialization component 630 may include one or more functional components that may be invoked when interface gateway 310 is operating in the real-time mode or the batch mode. Initialization component 630 may initialize an interface that will be used to process data in the real-time mode or the batch mode. For example, initialization component 630 may create the session logs for the processing of real-time or batch data and ensure, in the batch mode, that preprocess component 620 has moved the appropriate files to the target location.
- Preparation component 640 may include one or more functional components that may be invoked when interface gateway 310 is operating in the real-time mode or the batch mode.
- preparation component 640 may convert the input data (e.g., from user device 210 ) to a canonical structure (e.g., a canonical XML structure).
- preparation component 640 may convert the input data (e.g., from a retrieved file, a database, etc.) to a canonical structure (e.g., a canonical XML structure).
- preparation component 640 may invoke one or more services, in services component 425 , to perform the data conversions.
- Process component 650 may include one or more functional components that may be invoked when interface gateway 310 is operating in the real-time mode or the batch mode. Process component 650 may invoke one or more services, in services component 425 , to perform data translations, data transformations, etc. For example, process component 650 may invoke one or more services to retrieve the data in the canonical structure and translate/transform the data to the target structure. In addition, process component 650 may perform data edits/validations. Process component 650 may identify any records that fail the edits/validation check for notification and/or error correction purposes.
- Completion component 660 may include one or more functional components that may be invoked when interface gateway 310 is operating in the real-time mode or the batch mode. In addition, completion component 660 may be invoked when the interface execution succeeds and when the interface execution fails. For example, if the interface execution succeeds, completion component 660 may push the final data to the appropriate target system and notify master component 610 and notification engine 670 of the successful execution of the interface. In case of a failure, completion component 660 may notify master component 610 and notification engine 670 that the execution of the interface has failed.
- Notification engine 670 may include one or more functional components that notify appropriate personnel as to whether the execution of the interface has succeeded or failed. For example, notification engine 670 may receive notifications from completion component 660 and may send notifications to the appropriate personnel in response thereto and based on contact information from, for example, metadata database 440 . In one implementation, notification engine 670 may notify users of detected errors in response to data errors being logged in transactional database 445 .
- FIG. 6 illustrates exemplary functional components of orchestrator component 415
- orchestrator component 415 may include additional functional components, fewer functional components, different functional components, and/or differently arranged functional components than those illustrated in FIG. 6 .
- one or more functional components in FIG. 6 may perform one or more tasks described as being performed by one or more other functional components in FIG. 6 .
- FIG. 7 is a diagram of an exemplary portion of metadata database 440 .
- metadata database 440 may include, among other tables, an interface table 705 , a rule master table 710 , a rule details table 715 , a Rule Server Service (RSS) details table 720 , a server parameter table 725 , a service parameter table 730 , an application master table 735 , and a canonical master table 740 .
- RSS Rule Server Service
- Interface table 705 may be the main table in metadata database 440 .
- Interface table 705 may store details for each interface in interface gateway 310 .
- Each interface may be associated with a unique interface identifier (ID) in interface table 705 .
- This unique interface ID may provide a mapping from interface table 705 to rule master table 710 .
- Interface table 705 may also include information mapping to application master table 735 and canonical master table 740 .
- Interface table 705 may store an appl_id foreign key reference from application master table 735 and a canon_master_id foreign key reference from canonical master table 740 .
- Rule master table 710 may store information identifying the rules (e.g., business rules) to be used for implementing the interface, associated with the particular interface ID identified in rule master table 710 .
- rule master table 710 may identify the rules to be used by preprocess component 620 , initialization component 630 , preparation component 640 , process component 650 , and/or completion component 660 , and identify the order in which the rules are to be executed (e.g., a particular sequence or that some of the rules may be executed in parallel) for a particular interface.
- rule master table 710 may identify, for a particular interface, five rules (or steps) that are to be executed by process component 650 and indicate that these rules (or steps) may be executed in parallel.
- rule master table 710 may specify a particular process type (e.g., a process performed by initialization component 630 , a process performed by preparation component 640 , a process performed by process component 650 , and/or a process performed by completion component 660 ), a rule type (e.g., translation, transformation, edits/validations, etc.), and a run type (e.g., a regular run or an error run).
- a process type e.g., a process performed by initialization component 630 , a process performed by preparation component 640 , a process performed by process component 650 , and/or a process performed by completion component 660
- a rule type e.g., translation, transformation, edits/validations, etc.
- a run type e.g., a regular run or an error run.
- Each rule, in rule master table 710 may be associated with a rule ID.
- Rule details table 715 may be a child table of rule master table 710 .
- Rule details table 715 may store information identifying a list of steps to be performed for a given rule identified in rule master table 710 and information specifying the order that the steps are to be performed.
- Rule details table 715 may store a rule_id foreign key reference from rule master table 710 .
- RSS details table 720 may store information that maps a rule to a server and/or a service.
- RSS details table 720 may include a rule_det_id foreign key reference from rules detail table 720 , a server_id foreign key reference from server parameter table 725 , and a service_id foreign key reference from service parameter table 730 .
- Server parameter table 725 may act as the basic definition table for the different types of servers that may be used.
- server parameter table 725 may store server-related information (e.g., server name, server port, user details, context information, etc.) that is to be used for invoking different services.
- server parameter table 725 may store an appl_id foreign key reference from application master table 735 .
- Service parameter table 730 may store details (e.g., service names and scenarios) regarding all of the services associated with interface gateway 310 . Each service may be associated with a service ID. Service parameter table 730 may store an appl_id foreign key reference from application master table 735 .
- Application master table 735 may store information identifying the name of an application associated with interface gateway 310 and the corresponding code.
- Canonical master table 740 may store the details of the canonical structure and information that may be used to map canonical structures to the different applications associated with interface gateway 310 .
- Canonical master table 740 may store an appl_id foreign key reference from application master table 735 .
- FIG. 7 illustrates exemplary tables of metadata database 440
- metadata database 440 may include additional tables, fewer tables, different tables, and/or differently arranged tables than those illustrated in FIG. 7 . Additionally, or alternatively, one or more tables in FIG. 7 may store information described as being stored in one or more other tables in FIG. 7 .
- FIG. 8 is a diagram of exemplary functional components of portal 320 .
- portal 320 may include a configuration component 810 , a monitoring and audit component 820 , an error correction component 830 , and an administration component 840 .
- the functional components of portal 320 may be implemented using one or more components of device 500 of FIG. 5 .
- Configuration component 810 may include one or more functional components that allow a user to configure and/or reconfigure an interface at interface gateway 310 .
- configuration component 810 may provide one or more graphical user interfaces, to user device 210 , that allow a user to add a new interface to interface gateway 310 , to modify an existing interface associated with interface gateway 310 , to delete an existing interface associated with interface gateway 310 , to create/configure/reconfigure server, service, and canonical details for an interface, and/or to perform other functions relating to interfaces for interface gateway 310 .
- configuration component 810 may provide one or more graphical user interfaces that allow a user to specify, for example, a name for the new interface, provide parameters relating to the new interface (e.g., a description of the new interface, a type of the new interface, a group to be notified of events relating to execution of the new interface, and/or other parameters); identify rules for the interface and the order in which the rules are to be executed; identify, for each rule, a component of orchestrator component 415 (e.g., preprocess component 620 , initialization component 630 , preparation component 640 , process component 650 , or completion component 660 ) with which the rule is associated; identify, for each rule, a server and a service to be executed for the rule; provide details about the servers that will be used for executing the interface; provide details about the services that are to be used for executing the interface; provide details regarding the canonical structure to be used for the interface; and/or provide additional information.
- a component of orchestrator component 415 e.
- Monitoring and audit component 820 may include one or more functional components that provide a user with end-to-end status information as data is processed by interface gateway 310 .
- monitoring and audit component 820 may receive status information form real time monitor component 450 as data is being processed by interface gateway 310 and provide to a user, via one or more graphical user interfaces, the status of the data as the data is being processed (e.g., information identifying the component in interface gateway 310 that is currently processing the data).
- Monitoring and audit component 820 may also log data while interface gateway 310 processes the data, and provide the user with an audit report that, for example, provides information identifying each component in interface gateway 310 that has processing the data and, for each component, whether the processing was successful. Additional or other information may also be monitored and provided via monitoring and audit component 820 .
- Error correction component 830 may include one or more functional components that allow a user to correct errors that occurred during the processing of data by interface gateway 310 .
- Error correction component 830 may, for example, provide information to the user that identifies any errors that occurred during the processing of data at interface gateway 310 (e.g., based on information from transactional database 445 ) and allow the user to correct the errors.
- error correction component 830 may include a mass correction feature and a smart fix feature.
- the mass correction feature may allow the user to correct errors based on defined criteria.
- the smart fix feature may remember all the errors that have been previously fixed. The user may select the smart fixes to apply on future executions of the same interface.
- the smart fixes may include a default feature that may cause a value to default to a particular value, a substitution feature that substitutes a selected value with a new value, a truncate feature that may cause a particular field to take a particular value and ignore unwanted characters, and/or other types of features.
- Administration component 840 may include one or more functional components that allow a user (e.g., an administrator) to set privileges for accessing portal 320 .
- administration component 840 may provide one or more graphical user interfaces that allow a user to add new users and/or user groups, delete users and/or user groups, assign users to user groups, set access privileges for users and/or user groups, and/or to perform other functions relating to a user's or a user group's access to portal 320 .
- administration component 840 may include one or more functional components that perform authentication of users attempting to access portal 320 .
- administration component 840 may provide one or more graphical user interfaces to the user to authenticate the user prior to allowing the user to perform functions via portal 320 .
- administrator component 840 may allow additional graphical user interfaces to be provided to the user based on access privileges associated with the user.
- FIG. 8 illustrates exemplary functional components of portal 320
- portal 320 may include additional functional components, fewer functional components, different functional components, and/or differently arranged functional components than those illustrated in FIG. 8 .
- one or more functional components in FIG. 8 may perform one or more tasks described as being performed by one or more other functional components in FIG. 8 .
- FIG. 9 is a flow chart of an exemplary process for configuring an interface at interface gateway 310 .
- the processing of FIG. 9 may be performed by portal 320 .
- some or all of the processing described below may be performed by one or more components, including or excluding portal 320 .
- Process 900 may include authenticating a user (block 910 ).
- a user at user device 210
- portal 320 may access portal 320 (e.g., via network 240 ).
- portal 320 e.g., administration component 840
- portal 320 may deny the user access to portal 320 . Assume that portal 320 has properly authenticated the user.
- Process 900 may further include receiving configuration information for an interface (block 920 ).
- portal 320 may receive a request from the user to add a new interface to interface gateway 310 or modify an existing interface associated with interface gateway 310 .
- portal 320 may provide one or more graphical user interfaces to user device 210 to allow the user to define the new interface or modify an identified existing interface. Assume that the user desires to add a new interface to interface gateway 310 .
- Portal 320 may provide one or more graphical user interfaces to user device 210 to allow the user to specify, for example, a name for the new interface, provide parameters relating to the new interface (e.g., a description of the new interface, a type of the new interface, a group to be notified of events relating to execution of the new interface, and/or other parameters); identify rules for the interface and the order in which the rules are to be executed; identify, for each rule, a component of orchestrator component 415 (e.g., preprocess component 620 , initialization component 630 , preparation component 640 , process component 650 , or completion component 660 ) with which the rule is associated; identify, for each rule, a server and a service to be executed for the rule; provide details about the servers that will be used for executing the interface; provide details about the services that are to be used for executing the interface; provide details regarding the canonical structure to be used for the interface; and/or provide additional information.
- orchestrator component 415 e.g
- portal 320 may obtain, via one or more graphical user interfaces, information from the user to populate one or more of interface table 705 , rule master table 710 , rule details table 715 , RSS details table 720 , server parameter table 725 , service parameter table 730 , application master table 735 , canonical master table 740 , and/or other tables in metadata database 440 .
- Process 900 may additionally include storing the configuration information for the interface (block 930 ).
- portal 320 may store the configuration information in the appropriate tables of metadata database 440 .
- FIGS. 10A-10E are exemplary graphical user interfaces that may be provided to a user in adding a new interface to interface gateway 310 .
- a graphical user interface 1000 may allow a user to provide details regarding the new interface.
- graphical user interface 1000 may allow the user to specify a name of the new interface, a type of the new interface, and/or other information regarding the new interface.
- graphical user interface 1000 may allow the user to define rules to be associated with the new interface.
- graphical user interface 1000 may allow the user to specify, for each rule, the name of the rule to be associated with the new interface, the order in which the rule is to be executed in connection with other rules for the new interface, the run type of the new rule (e.g., basic run or error run), the identity of the component of orchestrator component 415 with which the rule is associated (referred to as the “process type” in FIG. 10A ), the rule type (e.g., translation, transformation, edits/validations, completion), and the type of service with which the rule is associated (e.g., ODI service, etc.).
- the run type of the new rule e.g., basic run or error run
- the identity of the component of orchestrator component 415 with which the rule is associated referred to as the “process type” in FIG. 10A
- the rule type e.g., translation, transformation, edits/validations, completion
- the type of service with which the rule is associated e.g., ODI service, etc.
- a graphical user interface 1010 may allow a user to provide details regarding a rule that has been specified in graphical user interface 1000 .
- graphical user interface 1010 may allow the user to specify details regarding one or more steps to be performed for a particular rule.
- graphical user interface 1010 may allow the user to specify a name for the step, an order in which the step is to be executed in relation to other identified steps, a rule invocation type (e.g., synchronous, asynchronous, or one way), an identifier of the server at which the step will be performed, an identifier of a service to be performed by the step, and/or other information.
- a rule invocation type e.g., synchronous, asynchronous, or one way
- a graphical user interface 1020 may allow a user to provide server parameters.
- graphical user interface 1020 may allow the user to specify a server (e.g., the name of a server), a server type (e.g., SAP, ODI, etc.), a host name for the server, a port of the server, and/or other information relating to a server used for executing the new interface.
- a graphical user interface 1030 may allow a user to provide service parameters.
- graphical user interface 1030 may allow the user to specify a service (e.g., the name of a service), a service version, a service type (e.g., JAVA, SAP, ODI, etc.), and/or other information relating to service parameters used for executing the new interface (such as call parameters for each service).
- a service e.g., the name of a service
- a service version e.g., JAVA, SAP, ODI, etc.
- service type e.g., JAVA, SAP, ODI, etc.
- a graphical user interface 1040 may allow a user to provide canonical structure details.
- graphical user interface 1040 may allow the user to specify a canonical structure (e.g., the name of a canonical structure), a description of the canonical structure, and/or other information relating to a canonical structure that may be associated with the new interface.
- the user may also import an existing canonical structure for use with the new interface.
- FIG. 11 is a flow chart of an exemplary process 1100 that may be performed by interface gateway 310 when operating in the real-time mode. In one implementation, the processing of FIG. 11 may be performed by interface gateway 310 . In another implementation, some or all of the processing described below may be performed by one or more components, including or excluding interface gateway 310 .
- Process 1100 may include receiving a real-time data request (block 1105 ).
- security component 405 may receive a real-time data request from a user device, such as user device 210 .
- the request may include an interface identifier, a user identifier, a source system, a source file name identifier, and information identifying a run mode.
- the interface identifier may identify the interface to be used for processing the request.
- the interface identifier may correspond to an identifier stored in interface table 705 of metadata database 440 .
- the user identifier may identify a user, a company, or some other entity.
- the source system identifier may identify a system from which the request is received.
- the source system identifier may correspond to a network address and/or some other type of identifier.
- the source file name identifier may identify the data to be processed by the interface.
- the data to be processed may accompany the request in the real-time mode.
- the run mode may indicate, for example, a basic run mode or an error mode.
- Process 1100 may also include authenticating the request (block 1110 ).
- security component 405 may use the user identifier, received in the request, to authenticate the user. Additionally, or alternatively, security component 405 may authenticate user device 210 or request information from the user or user device 210 to perform the authentication. If properly authenticated, security component 405 may pass the request to ISS component 410 . If not properly authenticated, security component 405 may deny the request.
- Process 1100 may additionally include converting data to a canonical structure (block 1115 ).
- ISS component 410 may receive the request and accompanying data from security component 405 and may convert the data to a canonical format.
- ISS component 410 may convert the data into a canonical XML format.
- ISS component 410 may store the converted data in a shared memory and forward the request to orchestrator component 415 .
- Process 1100 may include identifying an interface and the orchestrator components for processing the request (block 1120 ).
- orchestrator component 415 e.g., master component 610
- Master component 610 may receive the request (including the interface identifier) from ISS component 410 .
- Master component 610 may, using the interface identifier from the request, query interface table 705 , of metadata database 440 , to determine the orchestrator components (e.g., one or more of process component 650 or completion component 660 ) to be invoked for the request.
- master component 610 may also identify, from the tables of metadata database 440 , the rules that will be executed for the request, the servers and services that will be invoked for the request, and/or other information relating to the interface that will process the request.
- Process 1100 may further include processing the data using one or more of process component 650 or completion component 660 (block 1125 ). For example, if interface table 705 indicates that process component 650 is to be invoked, process component 650 may, based on the rules, perform a translation and/or transformation operation. For example, master component 610 may send the interface identifier and the process type to the translation/transformation process.
- the translation/transformation process may call the following services (from services component 425 ): a BPEL/Java/SAP service for performing the translation or transformation operation (e.g., a BPEL service may get the canonical XML from the shared memory and call the corresponding service to translate/transform the data); an adapter service; and/or an application service for applications (e.g., PeopleSoft, SAP, etc.); etc.
- process component 650 may perform an edits/validations operation. This operation may include causing any business rules/edits that may be required to be executed.
- these business rules/edits may exist across multiple applications, such as PeopleSoft, COBOL, ODI, etc.
- process component 650 may receive the interface identifier and process type from master component 610 and use this information to access one or more tables in metadata database 440 to obtain the details of the application/module that holds the edits definition and details.
- Process component 650 may invoke a WSDL service to the corresponding application/module to execute the business rules/edits. Any record that fails the edits/validations operation may be stored in the shared memory for sending a notification to the appropriate user.
- completion component 660 may, based on the rules, provide status information to notification engine 670 regarding the success and failure of the interface execution. If the interface execution is successful, completion component 660 may push the data into the appropriate target system and notify (e.g., via a success message) notification engine 670 , which may, in turn, notify the appropriate user(s) via electronic mail (e-mail) or in another manner. Completion component 660 may further notify master component 610 of the successful execution of the interface.
- completion component 660 may notify (e.g., via a failure message) notification engine 670 , which may, in turn, notify the appropriate user(s) via e-mail or in another manner. Completion component 660 may further notify master component 610 of the unsuccessful execution of the interface.
- Process 1100 may also include performing real-time monitoring of the execution of the interface (block 1130 ).
- real-time monitor 450 may track the status of the processing of the data. For example, real-time monitor 450 may track whether the data has been successfully processed by one or more components of orchestrator component 415 , whether the data has been unsuccessfully processed by one or more components of orchestrator component 415 , whether the data is currently being processed by one or more components of orchestrator component 415 , etc.
- Process 1100 may include storing the monitored information in transactional database 445 (block 1135 ).
- real-time monitor 450 (or another component or components of interface gateway 310 ) may store the status of data being processed in interface gateway 310 in transactional database 445 .
- real-time monitor 450 (or another component or components of interface gateway 310 ) may only store error information in transactional database 445 .
- the error information may include information identifying the data that experienced the error, information identifying the component of orchestrator component 415 that was processing the data when the error occurred, information identifying the service that was processing the data when the error occurred, the date/time when the error occurred, and/or other information.
- FIG. 12 is a flow chart of an exemplary process 1200 that may be performed by interface gateway 310 when operating in the batch mode. In one implementation, the processing of FIG. 12 may be performed by interface gateway 310 . In another implementation, some or all of the processing described below may be performed by one or more components, including or excluding interface gateway 310 .
- Process 1200 may include receiving a batch data request (block 1205 ).
- security component 405 may receive a batch data request from a scheduler, such as scheduler 220 .
- the request may include an interface identifier, a user identifier, a source system, a source file name identifier, and information identifying a run mode.
- the interface identifier may identify the interface to be used for processing the request.
- the interface identifier may correspond to an identifier stored in interface table 705 of metadata database 440 .
- the user identifier may identify a user, a company, or some other entity.
- the source system identifier may identify a scheduler from which the request is received.
- the source system identifier may correspond to a network address and/or some other type of identifier of scheduler 220 .
- the source file name identifier may identify a name of the data to be processed by the interface.
- the run mode may indicate, for example, a basic run mode or an error mode.
- Process 1200 may also include authenticating the request (block 1210 ). For example, security component 405 may use the user identifier, received in the request, to authenticate the user. Additionally, or alternatively, security component 405 may authenticate scheduler 220 or request information from scheduler 220 to perform the authentication. If properly authenticated, security component 405 may pass the request to orchestrator component 415 . If not properly authenticated, security component 405 may deny the request.
- authenticating the request block 1210 . For example, security component 405 may use the user identifier, received in the request, to authenticate the user. Additionally, or alternatively, security component 405 may authenticate scheduler 220 or request information from scheduler 220 to perform the authentication. If properly authenticated, security component 405 may pass the request to orchestrator component 415 . If not properly authenticated, security component 405 may deny the request.
- Process 1200 may include identifying an interface and the orchestrator components for processing the request (block 1215 ).
- orchestrator component 415 e.g., master component 610
- Master component 610 may receive the request (including the interface identifier) from security component 405 .
- Master component 610 may, using the interface identifier from the request, query interface table 705 , of metadata database 440 , to determine the orchestrator components (e.g., one or more of preprocess component 620 , initialization component 630 , preparation component 640 , process component 650 , or completion component 660 ) to be invoked for the request.
- master component 610 may also identify, from the tables of metadata database 440 , the rules that will be executed for the request, the servers and services that will be invoked for the request, and/or other information relating to the interface that will process the request.
- Process 1200 may further include processing the data using one or more of preprocess component 620 , initialization component 630 , preparation component 640 , process component 650 , or completion component 660 (block 1220 ). For example, if interface table 705 indicates that preprocess component 620 is to be invoked, preprocess component 620 may, based on the rules, receive the interface identifier and the file name from master component 610 . Preprocess component 620 may access metadata database 440 to obtain, for example, information identifying the source device at which the bulk data is located (e.g., an address of the source device), information identifying the target device to which the bulk data is to be transferred (e.g., an address of the target device), and information identifying the location of the bulk data on the source device.
- preprocess component 620 may, based on the rules, receive the interface identifier and the file name from master component 610 .
- Preprocess component 620 may access metadata database 440 to obtain, for example, information identifying the source device at which the bulk data is located (
- Preprocess component 620 may use this information to retrieve the bulk data from the source device and store the bulk data at the target device.
- preprocess component 620 may invoke an SFTP service to retrieve the bulk data, which may return success or failure as a parameter to master component 610 .
- initialization component 630 may, based on the rules, maintain session logs for the processing of the request. Initialization component 630 may also determine the name of the tables of metadata database 440 that are needed for processing the bulk data.
- preparation component 640 may, based on the rules, convert the data (e.g., from the file, a database, etc.) to a canonical XML format.
- the conversion may be based on information obtained from metadata database 440 and include one or more ODI service calls, one or more adapter calls (such as a database call, an FTP call, etc.), one or more Web Service Definition Language (WSDL) calls, one or more Java calls, and/or other types of calls.
- ODI service calls such as a database call, an FTP call, etc.
- WSDL Web Service Definition Language
- process component 650 may, based on the rules, perform a translation and/or transformation operation. For example, master component 610 may send the interface identifier and the process type to the translation/transformation process. Based on the data from one or more of the tables of metadata database 440 , the translation/transformation process may call the following services (from services component 425 ): an ODI/stored procedure service (e.g., the translation/transformation process may call the ODI service (or any service that is implemented in batch mode) to process the data from the canonical table); an adapter service; and/or an application service for applications (e.g., PeopleSoft, SAP, etc.); etc.
- an ODI/stored procedure service e.g., the translation/transformation process may call the ODI service (or any service that is implemented in batch mode) to process the data from the canonical table
- an adapter service e.g., the ODI service (or any service that is implemented in batch mode) to process the data from the canonical table
- process component 650 may perform an edits/validations operation. This operation may include causing any business rules/edits that may be required to be executed. In one implementation, these business rules/edits may exist across multiple applications, such as PeopleSoft, COBOL, ODI, etc. As an example, process component 650 may receive the interface identifier and process type from master component 610 and use this information to access one or more tables in metadata database 440 to obtain the details of the application/module that holds the edits definition and details. The application/module may access the shared memory to get the data for which the edits are to be performed, execute the edits, and store the data back in the shared memory.
- process component 650 may receive the interface identifier and process type from master component 610 and use this information to access one or more tables in metadata database 440 to obtain the details of the application/module that holds the edits definition and details.
- the application/module may access the shared memory to get the data for which the edits are to be performed, execute the edits, and store the data back in the
- completion component 660 may, based on the rules, provide status information to notification engine 670 regarding the success and failure of the interface execution. If the interface execution is successful, completion component 660 may push the data into the appropriate target system and notify (e.g., via a success message) notification engine 670 , which may, in turn, notify the appropriate user(s) via electronic mail (e-mail) or in another manner. Completion component 660 may further notify master component 610 of the successful execution of the interface.
- completion component 660 may notify (e.g., via a failure message) notification engine 670 , which may, in turn, notify the appropriate user(s) via e-mail or in another manner. Completion component 660 may further notify master component 610 of the unsuccessful execution of the interface.
- Process 1200 may also include performing monitoring of the execution of the interface (block 1225 ).
- real-time monitor 450 may track the status of the processing of the bulk data. For example, real-time monitor 450 may track whether the bulk data has been successfully processed by one or more components of orchestrator component 415 , whether the bulk data has been unsuccessfully processed by one or more components of orchestrator component 415 , whether the bulk data is currently being processed by one or more components of orchestrator component 415 , etc.
- Process 1200 may include storing the monitored information in transactional database 445 (block 1230 ).
- real-time monitor 450 (or another component or components of interface gateway 310 ) may store the status of processing of the bulk data in transactional database 445 .
- real-time monitor 450 (or another component or components of interface gateway 310 ) may only store error information in transactional database 445 .
- the error information may include information identifying the data in the bulk data (e.g., the particular record) that experienced the error, information identifying the component of orchestrator component 415 that was processing the data when the error occurred, information identifying the service that was processing the data when the error occurred, the date/time when the error occurred, and/or other information.
- FIG. 13 is a flow chart of an exemplary process for monitoring the execution of an interface.
- the processing of FIG. 13 may be performed by portal 320 .
- some or all of the processing described below may be performed by one or more components, including or excluding portal 320 .
- Process 1300 may include authenticating the user (block 1310 ).
- a user at user device 210 may access portal 320 .
- Portal 320 e.g., administration component 840
- portal 320 may deny the user access to portal 320 . Assume that portal 320 has properly authenticated the user.
- Process 1300 may also include receiving a request for process monitoring (block 1320 ).
- a user at user device 210 may cause user device 210 to transmit a request to monitor the processing of data by interface gateway 310 to portal 320 .
- Process 1300 may provide one or more graphical user interfaces to the user that allow the user to monitor the processing of data by interface gateway 310 (block 1330 ).
- portal 320 e.g., monitoring and audit component 820
- the current status may include information identifying one or more components of orchestrator component 415 that are currently processing the data and/or other information relating to the current status of the data or past status of the data.
- the status information may further include error information.
- FIG. 14 is a flow chart of an exemplary process for performing error correction.
- the processing of FIG. 14 may be performed by portal 320 .
- some or all of the processing described below may be performed by one or more components, including or excluding portal 320 .
- Process 1400 may include authenticating the user (block 1420 ).
- a user at user device 210 may access portal 320 .
- Portal 320 e.g., administration component 840
- portal 320 may deny the user access to portal 320 . Assume that portal 320 has properly authenticated the user.
- Process 1400 may also include receiving a request for performing error correction (block 1420 ).
- a user at user device 210 may cause user device 210 to transmit a request to perform error correction to portal 320 .
- Process 1400 may provide one or more graphical user interfaces to the user to allow the user to perform error correction (block 1430 ).
- portal 320 e.g., error correction component 830
- Error correction component 830 may dynamically connect to transactional database 445 to obtain the data from corresponding tables for that interface and instance.
- Error correction component 830 may provide one or more graphical user interfaces to the user to allow the user to correct errors that occurred, for example, when interface gateway 310 was operating in a bulk mode.
- Error correction component 830 may provide a mass correction feature and a smart fix feature.
- the mass correction feature may cause errors to be corrected that match selected criteria.
- the smart fix feature may remember all the errors that have been fixed previously and can apply those corrections to future executions of the same interface.
- the smart fix feature may allow, for example, for default values to be given to any particular column of a record that includes a null value, selected values to be replaced with another value, selected values to be truncated, and/or other operations.
- Error correction component 830 may further allow the user to generate a report that shows the errors that have occurred when executing a particular instance of an interface, cause the instance of the interface to be re-executed, and/or perform other operations.
- Implementations described herein provide a generic (one size fits all) interface gateway that can be used to implement any type of interface for various kinds of systems, such as ERP systems (e.g., SAP, PeopleSoft, etc.), Business Warehouse systems, Legacy systems, web services, business-to-business services, etc.
- the generic interface gateway can handle single payload requests, as well as batch request, where the payload is very big.
- the generic interface gateway may include a metadata-driven orchestration component that acts as the heart of the interface gateway. Users may configure an interface for the interface gateway by configuring the metadata-driven orchestration component to invoke whatever types of services are needed for processing the users' data.
- the orchestration component may read the metadata for the given interface to be executed and orchestrate the services in the defined order. The orchestration component may also decide whether the services can be triggered in sequential or parallel mode.
- a portal may be provided that allows for end-to-end status monitoring of the execution of an interface.
- the portal may also allow for dynamic error correction through, for example, a mass correction feature and/or a smart fix feature.
- a component may include hardware, such as a processor, ASIC, or FPGA, or a combination of hardware and software (e.g., a processor executing software).
Abstract
An interface gateway may receive a request including a first interface identifier. The interface gateway is associated with a group of interfaces, where each interface is associated with metadata that defines the interface. The metadata may include an interface identifier, information identifying services to be executed for the interface and an order in which the identified services are to be executed, and information identifying servers on which the identified services are implemented. The interface gateway may also identify, for the received request, one interface, of the group of interfaces, for processing the request based on the first interface identifier. The interface gateway may further process the received request using the one interface. When processing the received request, the interface gateway may execute the identified services on the identified servers according to the order, where the executing causes data, associated with the received request, to be converted from a source format to a target format.
Description
- This application claims priority to U.S. Provisional Patent Application No. 61/151,728, filed Feb. 11, 2009, the entire disclosure of which is incorporated by reference herein.
- A large enterprise may include a variety of data stores (e.g., operations support, enterprise resource planning (ERP), data warehouse (DW), enterprise performance management (EPM) applications, etc.), devices, and/or systems. However, the large enterprise may have difficulty providing end-to-end integration (e.g., communication integration, process integration between applications, etc.) between these data stores and/or devices given the diversity of the infrastructure. While there have been efforts to provide communication integration (e.g., an adapter framework) and process integration between specific applications (e.g., an Application Integration Architecture (AIA) framework), these solutions do not provide an infrastructure to integrate a heterogeneous system.
-
FIG. 1 is an exemplary overview of a service integration platform; -
FIG. 2 is a diagram of an exemplary environment according to implementations described herein; -
FIG. 3 is a diagram of exemplary components of a service integration platform ofFIG. 2 ; -
FIG. 4 is a diagram of exemplary components of an interface gateway ofFIG. 3 ; -
FIG. 5 is a diagram of exemplary components of a device; -
FIG. 6 is a diagram of exemplary functional components of an orchestrator ofFIG. 4 ; -
FIG. 7 is a diagram of an exemplary portion of a metadata database ofFIG. 4 ; -
FIG. 8 is a diagram of exemplary functional components of a portal ofFIG. 3 ; -
FIG. 9 is a flow chart of an exemplary process for configuring an interface at the interface gateway ofFIG. 3 ; -
FIGS. 10A-10E are diagrams of exemplary graphical user interfaces that may be provided in connection with the process described with respect toFIG. 9 ; -
FIG. 11 is a flow chart of an exemplary process that may be performed by an interface gateway when operating in a real-time mode; -
FIG. 12 is a flow chart of an exemplary process that may be performed by an interface gateway when operating in a batch mode; -
FIG. 13 is a flow chart of an exemplary process for monitoring the execution of an interface; and -
FIG. 14 is a flow chart of an exemplary process for performing error correction. - The following detailed description refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements. Also, the following detailed description does not limit the invention.
- Systems and methods, described herein, may provide service integration among multiple, heterogeneous systems. In an exemplary implementation, a service integration platform may provide the service integration based on a service-oriented architecture (SOA).
-
FIG. 1 is a diagram of anexemplary overview 100 of a service integration platform. Assume that data (denoted as data A) is to be transferred from the purchasing department of a business desires to the financial department of the business. The data may include any type of information, such as information that is needed by the financial department, information that a user in the purchasing department desires to send to a user in the financial department, and/or other types of information. Assume further that data A is in a first format and that the financial department requires that the information in data A be provided in a different format, shown as data B. - As described herein, a service integration platform may be provided that acts as a generic interface that can be used to interface various kinds of systems, such as Enterprise Resource Planning (ERP) systems, Business Warehouse systems, Legacy systems, web services, Business-to-Business (B2B) systems, etc. and/or data sources, such as BizFrame, VFrame, flat files, spreadsheets, etc., and/or other types of systems and/or data sources that may exist in a heterogeneous system. As such, the service integration platform may receive data in one type of format and from one type of system, convert the data to another type of format, and transfer the data to another type of system.
- Thus, in
overview 100, the user in the purchasing department, the user in the financial department, or another user (e.g., an administrator at the business) may configure an interface of the service integration platform to convert data from the purchasing department system to a desired format of the financial department system. Once the interface has been configured at the service integration platform, the user in the purchasing department may transfer data A to the service integration platform. The service integration platform may identify the appropriate interface for converting the information from data A, convert the information to the appropriate format used by the financial department system (to create data B), and transfer data B to the financial department system. -
FIG. 2 is a diagram of anexemplary environment 200 according to implementations described herein. As illustrated,environment 200 may include auser device 210, ascheduler 220, and aservice integration platform 230 that interconnect via anetwork 240. -
User device 210 may include one or more devices that can transmit a request to process data toservice integration platform 230 in a real-time mode. In addition,user device 210 may include one or more devices that allow a user to interact withservice integration platform 230. In one implementation,user device 210 may include a desktop computer, a laptop computer, a mainframe computer, a server, a handheld computing device (such as a cellular telephone, a personal digital assistant (PDA), etc.), and/or some other type of computational device.User device 210 may connect tonetwork 240 via a wired and/or wireless connection. - Scheduler 220 may include one or more devices that can transmit a request to process data to
service integration platform 230 in a batch mode. In one implementation,scheduler 220 may include a desktop computer, a laptop computer, a mainframe computer, a server, and/or some other type of computational device.Scheduler 220 may connect tonetwork 240 via a wired and/or wireless connection. -
Service integration platform 230 may include one or more devices that provide service integration for heterogeneous systems and/or data sources.Service integration platform 230 may include various interfaces that allow for transfer of data between different types of systems. In one implementation, each interface may have a one-to-one correspondence between a first system/data source and a second system/data source so as to provide data conversions between the first system/data source and the second system/data source. In addition,service integration platform 230 may include one or more devices that allow a user to interact withservice integration platform 230 to, for example, configure an interface onservice integration platform 230, monitor the processing of data atservice integration platform 230, perform error correction with respect to data being processed atservice integration platform 230, and/or perform other functions.Service integration platform 230 may include, for example, one or more mainframe computers, servers, desktop computers, laptop computers, and/or some other type of computational device. - As indicated above,
service integration platform 230 may operate in a real-time mode and a batch mode. In the real-time mode,service integration platform 230 may receive a request to process a single payload of data from a user device, such asuser device 210, identify an interface for processing the single payload, and process the single payload using the interface. In the batch mode,service integration platform 230 may receive a request to process a large payload from a scheduler, such asscheduler 220, identify an interface for processing the large payload, and process the large payload using the interface. Additional details regarding the real-time and batch modes are provided below in connection withFIGS. 11 and 12 , respectively. - Network 240 may include one or more networks of any type (wired and/or wireless). For example,
network 240 may include a local area network (LAN), a wide area network (WAN), a telephone network, such as a Public Switched Telephone Network (PSTN) or a cellular network, a satellite network, an intranet, the Internet, a data network, a private network, or a combination of networks. - Although
FIG. 2 illustrates exemplary components ofenvironment 200, in practice,environment 200 may include additional devices and/or networks, fewer devices and/or networks, different devices and/or networks, and/or differently arranged devices and/or networks than those illustrated inFIG. 2 . Additionally, or alternatively, one or more devices inFIG. 2 may perform one or more tasks described as being performed by one or more other devices inFIG. 2 . -
FIG. 3 is a diagram of exemplary components ofservice integration platform 230. As illustrated,service integration platform 230 may include aninterface gateway 310 and aportal 320. -
Interface gateway 310 may include one or more devices that act as the heart ofservice integration platform 230.Interface gateway 310 may provide the interfaces for processing data in real-time and batch modes.Interface gateway 310 may also provide monitoring services and error correction services.Interface gateway 310 may include, for example, one or more mainframe computers, servers, desktop computers, laptop computers, mainframe computers, and/or some other type of computational device. -
Portal 320 may include one or more devices that allow a user to interact withinterface gateway 310. For example, portal 320 may provide graphical user interfaces to a user device, such asuser device 210, that allow a user to configure an interface oninterface gateway 310, monitor the processing of data atinterface gateway 310, perform error correction with respect to data being processed atinterface gateway 310, and/or perform other functions. In addition, portal 320 may authenticate a user prior to providing the graphical user interfaces. In one implementation, the graphical user interfaces may be provided as web pages.Portal 320 may include, for example, one or more mainframe computers, servers, desktop computers, laptop computers, and/or some other type of computational device. - Although
FIG. 3 illustrates exemplary components ofservice integration platform 230, in practice,service integration platform 230 may include additional components, fewer components, different components, and/or differently arranged components than those illustrated inFIG. 3 . Additionally, or alternatively, one component inFIG. 3 may perform one or more tasks described as being performed by another component inFIG. 3 . -
FIG. 4 is a diagram of exemplary components ofinterface gateway 310. As illustrated,interface gateway 310 may include asecurity component 405, an Integrated Specific Service (ISS)component 410, anorchestrator component 415, aservice adapter component 420, aservices component 425, acache component 430, anapplication database component 435, ametadata database 440, atransactional database 445, and a realtime monitor component 450. -
Security component 405 may include one or more devices that authenticate a received request. The authentication may include authenticating the device from which the request is received and/or the user with which the request is associated. For example,security component 405 may receive a real-time request fromuser device 210 and may, in response to receiving the request, authenticateuser device 210. For example, the real-time request may include information identifyinguser device 210 andsecurity component 405 may use that information to authenticateuser device 210. Additionally, or alternatively,security component 405 may send a request for information touser device 210 and may use information returned from that request to authenticateuser device 210.Security component 405 may further receive a batch request fromscheduler 220 and may, in response to receiving the request, authenticatescheduler 220. For example, the batch request may includeinformation identifying scheduler 220 andsecurity component 405 may use that information to authenticatescheduler 220. Additionally, or alternatively,security component 405 may send a request for information toscheduler 220 and may use information returned from that request to authenticatescheduler 220.Security component 405 may transfer received requests toISS component 410 in the real-time mode and/or toorchestrator component 415 in the batch mode. -
ISS component 410 may include one or more devices to convert received data, in the real-time mode, to a canonical structure. In one implementation, the canonical structure may include a canonical eXtensible Markup Language (XML) structure. Thus, for example, if the incoming data is in the form of an XML document, an Intermediate Document (IDoc), a string, etc.,ISS component 410 may convert the incoming data (regardless of the format) to the canonical XML structure. Other types of canonical structures can alternatively be used.ISS component 410 may transfer the data in the canonical structure toorchestrator 415. - In one exemplary implementation,
ISS component 410 may include a data router and a group of translators, where each translator translates a different data format to the canonical structure. The data router may receive data, detect the format of the received data, identify, based on the detected format, the appropriate translator for translating the data, and route the data to the identified translator. The translator may then translate the received data to the canonical structure. To facilitate the requesting router's ability to detect the format of received data, in some implementations, users may use different naming conventions for naming the data, where the different naming conventions identify the format of the data. In these implementations, the data router may use the naming convention of incoming data to identify the appropriate translator to which to route the data. -
Orchestrator component 415 may include one or more devices that act as the main component ofinterface gateway 310.Orchestrator component 415 may determine, based on data frommetadata database 440, the other components ofinterface gateway 310 that need to be invoked to complete an interface. In one implementation,orchestrator component 415 may be implemented using Business Process Execution Language (BPEL).Orchestrator 415 may operate in an asynchronous manner in the batch mode and a synchronous and/or asynchronous manner in the real-time mode. Further details regardingorchestrator component 415 are provided below with respect toFIG. 6 . -
Service adapter component 420 may include one or more devices that act as a service implementation channel to access the services inservices component 425. In one implementation,service adapter component 420 may include a group of adapters that allow for different types of services to be invoked. For example, the group of adapters may include one or more Oracle Data Integration (ODI) adapters, SAP adapters, database adapters, legacy adapters, File Transfer Protocol (FTP) adapters, and/or other types of adapters.Service adapter component 420 may enable a service-oriented architecture to exist ininterface gateway 310, where new services may be added toservices component 425 without affecting the rest ofinterface gateway 310. -
Services component 425 may include one or more devices that provide services for processing data. The services may include process monitoring services, masking engine services, data services (e.g., that allow connectivity to databases), data error handling services, correction and recycling services, notification services, error handling services, Secure File Transfer Protocol (SFTP) services, and/or other types of services. The services may be implemented as ODI services, SAP services, Java Web Services, Unix shell script, and/or other types of services. -
Cache component 430 may include one or more devices that may store data. In one implementation,cache component 430 may cache data (e.g., in one or more reference tables) fromapplication database 435 that may be needed byservices component 425. In this way, data may be available toservices component 425 faster than if the data is retrieved fromapplication database 435. -
Application database 435 may include one or more devices that store data that may be accessed byservices component 425. As indicated above,cache component 430 may retrieve data, fromapplication database 435, which may be needed byservices component 425. -
Metadata database 440 may include one or more devices that store metadata that define the interfaces ofinterface gateway 310. For example,metadata database 440 may store the parameters of each interface with whichinterface gateway 310 is associated, along with information identifying the rules to be executed for each interface, the servers that need to be called for each interface, the services to be executed for each interface, and/or other information needed for executing an interface. While one metadata database is shown, it will be appreciated thatmetadata database 440 may consist of multiple databases stored locally atinterface gateway 310, or stored at one or more different and possibly remote locations. Additional details regardingmetadata database 440 are provided below with respect toFIG. 7 . -
Transactional database 445 may include one or more devices that store information relating to the processing of data byinterface gateway 310. In one implementation,transactional database 445 may store information relating to errors that occurred while processing data byinterface gateway 310. For example,transactional database 445 may maintain error tables that store data errors that were captured while processing the data ininterface gateway 310. While one transactional database is shown, it will be appreciated thattransactional database 445 may consist of multiple databases stored locally atinterface gateway 310, or stored at one or more different and possibly remote locations. - Real
time monitor component 450 may include one or more devices that may monitor the operation oforchestrator component 415 and/or other components ofinterface gateway 310. For example,orchestrator component 415 may transmit, in real time, status information to realtime monitor component 450. Realtime monitor component 450 may receive this status information and, for example, provide the status information toportal 320 to allow a user to track the latest status of the execution of data using the interfaces ofinterface gateway 310. - Although
FIG. 4 illustrates exemplary components ofinterface gateway 310, in practice,interface gateway 310 may include additional components, fewer components, different components, and/or differently arranged components than those illustrated inFIG. 4 . Additionally, or alternatively, one or more components inFIG. 4 may perform one or more tasks described as being performed by one or more other components inFIG. 4 . -
FIG. 5 is a diagram of exemplary components of adevice 500 that may correspond to one or more of the devices or components depicted inFIGS. 2-4 . For example,device 500 may correspond touser device 210,scheduler 220, one or more components ofinterface gateway 310, and/or one or more components ofportal 320. As illustrated,device 500 may include aprocessing system 505, amemory 510, aninput component 515, anoutput component 520, and acommunication interface 525. -
Processing system 505 may include one or more processors, microprocessors, and/or another type of component that may interpret and execute instructions. In some implementations,processing system 505 may include one or more application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. -
Memory 510 may include a random access memory (RAM), a dynamic random access memory (DRAM), a ferroelectric random access memory (FRAM), a read only memory (ROM), a programmable read only memory (PROM), a flash memory, and/or some other type of memory.Memory 510 may additionally, or alternatively, include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid state disk, etc.) or some other type of computer-readable medium, along with a corresponding drive. A computer-readable medium may correspond to, for example, a physical memory device or a logical memory device. A logical memory device may include memory space within a single physical memory device or spread across multiple physical memory devices residing on one or more devices. -
Input component 515 may permit an operator to input information intodevice 500. For example,input component 515 may include a keyboard, a button, a switch, a knob, a touchpad, a display, and/or some other type of input component.Output component 520 may permitdevice 500 to output information to the operator. For example,output component 520 may include a display, light emitting diodes (LEDs), a speaker, and/or some other type of output component. -
Communication interface 525 may permitdevice 500 to communicate with other devices, networks, and/or systems. For example,communication interface 520 may include some type of wireless and/or wired interface. - As described herein,
device 500 may perform certain operations in response toprocessing system 505 executing software instructions contained in a computer-readable medium, such asmemory 510. The software instructions may be read intomemory 510 from another computer-readable medium or from another device viacommunication interface 525. The software instructions contained inmemory 510 may causeprocessing system 505 to perform processes described herein. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software. - Although
FIG. 5 illustrates exemplary components ofdevice 500, in practice,device 500 may include additional components, fewer components, different components, and/or differently arranged components than those illustrated inFIG. 5 . Additionally, or alternatively, one or more components inFIG. 5 may perform one or more tasks described as being performed by one or more other components inFIG. 5 . -
FIG. 6 is a diagram of exemplary functional components oforchestrator component 415. As illustrated inFIG. 6 ,orchestrator component 415 may include amaster component 610, apreprocess component 620, aninitialization component 630, apreparation component 640, aprocess component 650, acompletion component 660, and anotification engine 670. The functional components oforchestrator component 415 may be implemented using one or more components ofdevice 500 ofFIG. 5 . -
Master component 610 may include one or more functional components that may be invoked wheninterface gateway 310 is operating in the real-time mode or the batch mode.Master component 610 may receive information from ISS 410 (in the real-time mode) or security component 405 (in the batch mode), identify an interface, and invoke the appropriate one or ones ofpreprocessing component 620,initialization component 630,preparation component 640,process component 650, andcompletion component 660 for executing the identified interface. In this way,master component 610 may control the execution of an interface for processing received data. In one implementation,master component 610 may be implemented using BPEL. -
Preprocess component 620 may include one or more functional components that may be invoked wheninterface gateway 310 is operating in the batch mode.Preprocess component 620 may receive information identifying one or more files to be processed and the location of the one or more files.Preprocess component 620 may use the received information to retrieve the one or more files from the location and store the one or more files in a target location. -
Initialization component 630 may include one or more functional components that may be invoked wheninterface gateway 310 is operating in the real-time mode or the batch mode.Initialization component 630 may initialize an interface that will be used to process data in the real-time mode or the batch mode. For example,initialization component 630 may create the session logs for the processing of real-time or batch data and ensure, in the batch mode, that preprocesscomponent 620 has moved the appropriate files to the target location. -
Preparation component 640 may include one or more functional components that may be invoked wheninterface gateway 310 is operating in the real-time mode or the batch mode. In the real-time mode,preparation component 640 may convert the input data (e.g., from user device 210) to a canonical structure (e.g., a canonical XML structure). In the batch mode,preparation component 640 may convert the input data (e.g., from a retrieved file, a database, etc.) to a canonical structure (e.g., a canonical XML structure). In the real-time and batch modes,preparation component 640 may invoke one or more services, inservices component 425, to perform the data conversions. -
Process component 650 may include one or more functional components that may be invoked wheninterface gateway 310 is operating in the real-time mode or the batch mode.Process component 650 may invoke one or more services, inservices component 425, to perform data translations, data transformations, etc. For example,process component 650 may invoke one or more services to retrieve the data in the canonical structure and translate/transform the data to the target structure. In addition,process component 650 may perform data edits/validations.Process component 650 may identify any records that fail the edits/validation check for notification and/or error correction purposes. -
Completion component 660 may include one or more functional components that may be invoked wheninterface gateway 310 is operating in the real-time mode or the batch mode. In addition,completion component 660 may be invoked when the interface execution succeeds and when the interface execution fails. For example, if the interface execution succeeds,completion component 660 may push the final data to the appropriate target system and notifymaster component 610 andnotification engine 670 of the successful execution of the interface. In case of a failure,completion component 660 may notifymaster component 610 andnotification engine 670 that the execution of the interface has failed. -
Notification engine 670 may include one or more functional components that notify appropriate personnel as to whether the execution of the interface has succeeded or failed. For example,notification engine 670 may receive notifications fromcompletion component 660 and may send notifications to the appropriate personnel in response thereto and based on contact information from, for example,metadata database 440. In one implementation,notification engine 670 may notify users of detected errors in response to data errors being logged intransactional database 445. - Although
FIG. 6 illustrates exemplary functional components oforchestrator component 415, in practice,orchestrator component 415 may include additional functional components, fewer functional components, different functional components, and/or differently arranged functional components than those illustrated inFIG. 6 . Additionally, or alternatively, one or more functional components inFIG. 6 may perform one or more tasks described as being performed by one or more other functional components inFIG. 6 . -
FIG. 7 is a diagram of an exemplary portion ofmetadata database 440. As shown,metadata database 440 may include, among other tables, an interface table 705, a rule master table 710, a rule details table 715, a Rule Server Service (RSS) details table 720, a server parameter table 725, a service parameter table 730, an application master table 735, and a canonical master table 740. - Interface table 705 may be the main table in
metadata database 440. Interface table 705 may store details for each interface ininterface gateway 310. Each interface may be associated with a unique interface identifier (ID) in interface table 705. This unique interface ID may provide a mapping from interface table 705 to rule master table 710. Interface table 705 may also include information mapping to application master table 735 and canonical master table 740. Interface table 705 may store an appl_id foreign key reference from application master table 735 and a canon_master_id foreign key reference from canonical master table 740. - Rule master table 710 may store information identifying the rules (e.g., business rules) to be used for implementing the interface, associated with the particular interface ID identified in rule master table 710. In one implementation, rule master table 710 may identify the rules to be used by
preprocess component 620,initialization component 630,preparation component 640,process component 650, and/orcompletion component 660, and identify the order in which the rules are to be executed (e.g., a particular sequence or that some of the rules may be executed in parallel) for a particular interface. For example, rule master table 710 may identify, for a particular interface, five rules (or steps) that are to be executed byprocess component 650 and indicate that these rules (or steps) may be executed in parallel. Among other things, rule master table 710 may specify a particular process type (e.g., a process performed byinitialization component 630, a process performed bypreparation component 640, a process performed byprocess component 650, and/or a process performed by completion component 660), a rule type (e.g., translation, transformation, edits/validations, etc.), and a run type (e.g., a regular run or an error run). Each rule, in rule master table 710, may be associated with a rule ID. - Rule details table 715 may be a child table of rule master table 710. Rule details table 715 may store information identifying a list of steps to be performed for a given rule identified in rule master table 710 and information specifying the order that the steps are to be performed. Rule details table 715 may store a rule_id foreign key reference from rule master table 710.
- RSS details table 720 may store information that maps a rule to a server and/or a service. Thus, RSS details table 720 may include a rule_det_id foreign key reference from rules detail table 720, a server_id foreign key reference from server parameter table 725, and a service_id foreign key reference from service parameter table 730.
- Server parameter table 725 may act as the basic definition table for the different types of servers that may be used. For example, server parameter table 725 may store server-related information (e.g., server name, server port, user details, context information, etc.) that is to be used for invoking different services. Server parameter table 725 may store an appl_id foreign key reference from application master table 735.
- Service parameter table 730 may store details (e.g., service names and scenarios) regarding all of the services associated with
interface gateway 310. Each service may be associated with a service ID. Service parameter table 730 may store an appl_id foreign key reference from application master table 735. - Application master table 735 may store information identifying the name of an application associated with
interface gateway 310 and the corresponding code. - Canonical master table 740 may store the details of the canonical structure and information that may be used to map canonical structures to the different applications associated with
interface gateway 310. Canonical master table 740 may store an appl_id foreign key reference from application master table 735. - Although
FIG. 7 illustrates exemplary tables ofmetadata database 440, in practice,metadata database 440 may include additional tables, fewer tables, different tables, and/or differently arranged tables than those illustrated inFIG. 7 . Additionally, or alternatively, one or more tables inFIG. 7 may store information described as being stored in one or more other tables inFIG. 7 . -
FIG. 8 is a diagram of exemplary functional components ofportal 320. As illustrated, portal 320 may include aconfiguration component 810, a monitoring andaudit component 820, anerror correction component 830, and anadministration component 840. The functional components ofportal 320 may be implemented using one or more components ofdevice 500 ofFIG. 5 . -
Configuration component 810 may include one or more functional components that allow a user to configure and/or reconfigure an interface atinterface gateway 310. For example,configuration component 810 may provide one or more graphical user interfaces, touser device 210, that allow a user to add a new interface tointerface gateway 310, to modify an existing interface associated withinterface gateway 310, to delete an existing interface associated withinterface gateway 310, to create/configure/reconfigure server, service, and canonical details for an interface, and/or to perform other functions relating to interfaces forinterface gateway 310. For example,configuration component 810 may provide one or more graphical user interfaces that allow a user to specify, for example, a name for the new interface, provide parameters relating to the new interface (e.g., a description of the new interface, a type of the new interface, a group to be notified of events relating to execution of the new interface, and/or other parameters); identify rules for the interface and the order in which the rules are to be executed; identify, for each rule, a component of orchestrator component 415 (e.g.,preprocess component 620,initialization component 630,preparation component 640,process component 650, or completion component 660) with which the rule is associated; identify, for each rule, a server and a service to be executed for the rule; provide details about the servers that will be used for executing the interface; provide details about the services that are to be used for executing the interface; provide details regarding the canonical structure to be used for the interface; and/or provide additional information. - Monitoring and
audit component 820 may include one or more functional components that provide a user with end-to-end status information as data is processed byinterface gateway 310. For example, monitoring andaudit component 820 may receive status information form realtime monitor component 450 as data is being processed byinterface gateway 310 and provide to a user, via one or more graphical user interfaces, the status of the data as the data is being processed (e.g., information identifying the component ininterface gateway 310 that is currently processing the data). Monitoring andaudit component 820 may also log data whileinterface gateway 310 processes the data, and provide the user with an audit report that, for example, provides information identifying each component ininterface gateway 310 that has processing the data and, for each component, whether the processing was successful. Additional or other information may also be monitored and provided via monitoring andaudit component 820. -
Error correction component 830 may include one or more functional components that allow a user to correct errors that occurred during the processing of data byinterface gateway 310.Error correction component 830 may, for example, provide information to the user that identifies any errors that occurred during the processing of data at interface gateway 310 (e.g., based on information from transactional database 445) and allow the user to correct the errors. In one implementation,error correction component 830 may include a mass correction feature and a smart fix feature. The mass correction feature may allow the user to correct errors based on defined criteria. The smart fix feature may remember all the errors that have been previously fixed. The user may select the smart fixes to apply on future executions of the same interface. The smart fixes may include a default feature that may cause a value to default to a particular value, a substitution feature that substitutes a selected value with a new value, a truncate feature that may cause a particular field to take a particular value and ignore unwanted characters, and/or other types of features. -
Administration component 840 may include one or more functional components that allow a user (e.g., an administrator) to set privileges for accessingportal 320. For example,administration component 840 may provide one or more graphical user interfaces that allow a user to add new users and/or user groups, delete users and/or user groups, assign users to user groups, set access privileges for users and/or user groups, and/or to perform other functions relating to a user's or a user group's access toportal 320. In addition,administration component 840 may include one or more functional components that perform authentication of users attempting to access portal 320. For example, in response to auser accessing portal 320,administration component 840 may provide one or more graphical user interfaces to the user to authenticate the user prior to allowing the user to perform functions viaportal 320. When a user has been properly authenticated,administrator component 840 may allow additional graphical user interfaces to be provided to the user based on access privileges associated with the user. - Although
FIG. 8 illustrates exemplary functional components ofportal 320, in practice, portal 320 may include additional functional components, fewer functional components, different functional components, and/or differently arranged functional components than those illustrated inFIG. 8 . Additionally, or alternatively, one or more functional components inFIG. 8 may perform one or more tasks described as being performed by one or more other functional components inFIG. 8 . -
FIG. 9 is a flow chart of an exemplary process for configuring an interface atinterface gateway 310. In one implementation, the processing ofFIG. 9 may be performed byportal 320. In another implementation, some or all of the processing described below may be performed by one or more components, including or excludingportal 320. -
Process 900 may include authenticating a user (block 910). For example, a user (at user device 210) may access portal 320 (e.g., via network 240). In response, portal 320 (e.g., administration component 840) may authenticate the user by, for example, requesting a user identifier and password from the user, receiving the user identifier and password from the user, and comparing the received user identifier and password to a stored list of user identifiers and passwords. If the received user identifier and password match a user identifier and password in the list of user identifiers and passwords, portal 320 may determine that the user has been properly authenticated and may provide the user with access toportal 320. Alternatively, if the received user identifier and password do not match a user identifier and password in the list of user identifiers and passwords, portal 320 may deny the user access toportal 320. Assume that portal 320 has properly authenticated the user. -
Process 900 may further include receiving configuration information for an interface (block 920). For example, portal 320 may receive a request from the user to add a new interface tointerface gateway 310 or modify an existing interface associated withinterface gateway 310. In response, portal 320 may provide one or more graphical user interfaces touser device 210 to allow the user to define the new interface or modify an identified existing interface. Assume that the user desires to add a new interface tointerface gateway 310.Portal 320 may provide one or more graphical user interfaces touser device 210 to allow the user to specify, for example, a name for the new interface, provide parameters relating to the new interface (e.g., a description of the new interface, a type of the new interface, a group to be notified of events relating to execution of the new interface, and/or other parameters); identify rules for the interface and the order in which the rules are to be executed; identify, for each rule, a component of orchestrator component 415 (e.g.,preprocess component 620,initialization component 630,preparation component 640,process component 650, or completion component 660) with which the rule is associated; identify, for each rule, a server and a service to be executed for the rule; provide details about the servers that will be used for executing the interface; provide details about the services that are to be used for executing the interface; provide details regarding the canonical structure to be used for the interface; and/or provide additional information. In one implementation,portal 320 may obtain, via one or more graphical user interfaces, information from the user to populate one or more of interface table 705, rule master table 710, rule details table 715, RSS details table 720, server parameter table 725, service parameter table 730, application master table 735, canonical master table 740, and/or other tables inmetadata database 440. -
Process 900 may additionally include storing the configuration information for the interface (block 930). For example, portal 320 may store the configuration information in the appropriate tables ofmetadata database 440. -
FIGS. 10A-10E are exemplary graphical user interfaces that may be provided to a user in adding a new interface tointerface gateway 310. InFIG. 10A , agraphical user interface 1000 may allow a user to provide details regarding the new interface. For example,graphical user interface 1000 may allow the user to specify a name of the new interface, a type of the new interface, and/or other information regarding the new interface. Moreover,graphical user interface 1000 may allow the user to define rules to be associated with the new interface. For example,graphical user interface 1000 may allow the user to specify, for each rule, the name of the rule to be associated with the new interface, the order in which the rule is to be executed in connection with other rules for the new interface, the run type of the new rule (e.g., basic run or error run), the identity of the component oforchestrator component 415 with which the rule is associated (referred to as the “process type” inFIG. 10A ), the rule type (e.g., translation, transformation, edits/validations, completion), and the type of service with which the rule is associated (e.g., ODI service, etc.). - In
FIG. 10B , agraphical user interface 1010 may allow a user to provide details regarding a rule that has been specified ingraphical user interface 1000. For example,graphical user interface 1010 may allow the user to specify details regarding one or more steps to be performed for a particular rule. InFIG. 10B ,graphical user interface 1010 may allow the user to specify a name for the step, an order in which the step is to be executed in relation to other identified steps, a rule invocation type (e.g., synchronous, asynchronous, or one way), an identifier of the server at which the step will be performed, an identifier of a service to be performed by the step, and/or other information. - In
FIG. 10C , agraphical user interface 1020 may allow a user to provide server parameters. For example,graphical user interface 1020 may allow the user to specify a server (e.g., the name of a server), a server type (e.g., SAP, ODI, etc.), a host name for the server, a port of the server, and/or other information relating to a server used for executing the new interface. - In
FIG. 10D , agraphical user interface 1030 may allow a user to provide service parameters. For example,graphical user interface 1030 may allow the user to specify a service (e.g., the name of a service), a service version, a service type (e.g., JAVA, SAP, ODI, etc.), and/or other information relating to service parameters used for executing the new interface (such as call parameters for each service). - In
FIG. 10E , agraphical user interface 1040 may allow a user to provide canonical structure details. For example,graphical user interface 1040 may allow the user to specify a canonical structure (e.g., the name of a canonical structure), a description of the canonical structure, and/or other information relating to a canonical structure that may be associated with the new interface. The user may also import an existing canonical structure for use with the new interface. -
FIG. 11 is a flow chart of anexemplary process 1100 that may be performed byinterface gateway 310 when operating in the real-time mode. In one implementation, the processing ofFIG. 11 may be performed byinterface gateway 310. In another implementation, some or all of the processing described below may be performed by one or more components, including or excludinginterface gateway 310. -
Process 1100 may include receiving a real-time data request (block 1105). For example,security component 405 may receive a real-time data request from a user device, such asuser device 210. The request may include an interface identifier, a user identifier, a source system, a source file name identifier, and information identifying a run mode. The interface identifier may identify the interface to be used for processing the request. For example, the interface identifier may correspond to an identifier stored in interface table 705 ofmetadata database 440. The user identifier may identify a user, a company, or some other entity. The source system identifier may identify a system from which the request is received. For example, the source system identifier may correspond to a network address and/or some other type of identifier. The source file name identifier may identify the data to be processed by the interface. In one implementation, the data to be processed may accompany the request in the real-time mode. The run mode may indicate, for example, a basic run mode or an error mode. -
Process 1100 may also include authenticating the request (block 1110). For example,security component 405 may use the user identifier, received in the request, to authenticate the user. Additionally, or alternatively,security component 405 may authenticateuser device 210 or request information from the user oruser device 210 to perform the authentication. If properly authenticated,security component 405 may pass the request toISS component 410. If not properly authenticated,security component 405 may deny the request. -
Process 1100 may additionally include converting data to a canonical structure (block 1115). For example,ISS component 410 may receive the request and accompanying data fromsecurity component 405 and may convert the data to a canonical format. In one implementation,ISS component 410 may convert the data into a canonical XML format.ISS component 410 may store the converted data in a shared memory and forward the request toorchestrator component 415. -
Process 1100 may include identifying an interface and the orchestrator components for processing the request (block 1120). For example, orchestrator component 415 (e.g., master component 610) may receive the request (including the interface identifier) fromISS component 410.Master component 610 may, using the interface identifier from the request, query interface table 705, ofmetadata database 440, to determine the orchestrator components (e.g., one or more ofprocess component 650 or completion component 660) to be invoked for the request. In addition,master component 610 may also identify, from the tables ofmetadata database 440, the rules that will be executed for the request, the servers and services that will be invoked for the request, and/or other information relating to the interface that will process the request. -
Process 1100 may further include processing the data using one or more ofprocess component 650 or completion component 660 (block 1125). For example, if interface table 705 indicates thatprocess component 650 is to be invoked,process component 650 may, based on the rules, perform a translation and/or transformation operation. For example,master component 610 may send the interface identifier and the process type to the translation/transformation process. Based on the data from one or more of the tables ofmetadata database 440, the translation/transformation process may call the following services (from services component 425): a BPEL/Java/SAP service for performing the translation or transformation operation (e.g., a BPEL service may get the canonical XML from the shared memory and call the corresponding service to translate/transform the data); an adapter service; and/or an application service for applications (e.g., PeopleSoft, SAP, etc.); etc. Additionally, or alternatively,process component 650 may perform an edits/validations operation. This operation may include causing any business rules/edits that may be required to be executed. In one implementation, these business rules/edits may exist across multiple applications, such as PeopleSoft, COBOL, ODI, etc. As an example,process component 650 may receive the interface identifier and process type frommaster component 610 and use this information to access one or more tables inmetadata database 440 to obtain the details of the application/module that holds the edits definition and details.Process component 650 may invoke a WSDL service to the corresponding application/module to execute the business rules/edits. Any record that fails the edits/validations operation may be stored in the shared memory for sending a notification to the appropriate user. - If interface table 705 indicates that
completion component 660 is to be invoked,completion component 660 may, based on the rules, provide status information tonotification engine 670 regarding the success and failure of the interface execution. If the interface execution is successful,completion component 660 may push the data into the appropriate target system and notify (e.g., via a success message)notification engine 670, which may, in turn, notify the appropriate user(s) via electronic mail (e-mail) or in another manner.Completion component 660 may further notifymaster component 610 of the successful execution of the interface. If, on the other hand, the interface execution fails,completion component 660 may notify (e.g., via a failure message)notification engine 670, which may, in turn, notify the appropriate user(s) via e-mail or in another manner.Completion component 660 may further notifymaster component 610 of the unsuccessful execution of the interface. -
Process 1100 may also include performing real-time monitoring of the execution of the interface (block 1130). In one implementation, while the appropriate interface ofinterface gateway 310 is being executing to process the data, real-time monitor 450 may track the status of the processing of the data. For example, real-time monitor 450 may track whether the data has been successfully processed by one or more components oforchestrator component 415, whether the data has been unsuccessfully processed by one or more components oforchestrator component 415, whether the data is currently being processed by one or more components oforchestrator component 415, etc. -
Process 1100 may include storing the monitored information in transactional database 445 (block 1135). For example, real-time monitor 450 (or another component or components of interface gateway 310) may store the status of data being processed ininterface gateway 310 intransactional database 445. In one implementation, real-time monitor 450 (or another component or components of interface gateway 310) may only store error information intransactional database 445. The error information may include information identifying the data that experienced the error, information identifying the component oforchestrator component 415 that was processing the data when the error occurred, information identifying the service that was processing the data when the error occurred, the date/time when the error occurred, and/or other information. -
FIG. 12 is a flow chart of anexemplary process 1200 that may be performed byinterface gateway 310 when operating in the batch mode. In one implementation, the processing ofFIG. 12 may be performed byinterface gateway 310. In another implementation, some or all of the processing described below may be performed by one or more components, including or excludinginterface gateway 310. -
Process 1200 may include receiving a batch data request (block 1205). For example,security component 405 may receive a batch data request from a scheduler, such asscheduler 220. The request may include an interface identifier, a user identifier, a source system, a source file name identifier, and information identifying a run mode. The interface identifier may identify the interface to be used for processing the request. For example, the interface identifier may correspond to an identifier stored in interface table 705 ofmetadata database 440. The user identifier may identify a user, a company, or some other entity. The source system identifier may identify a scheduler from which the request is received. For example, the source system identifier may correspond to a network address and/or some other type of identifier ofscheduler 220. The source file name identifier may identify a name of the data to be processed by the interface. The run mode may indicate, for example, a basic run mode or an error mode. -
Process 1200 may also include authenticating the request (block 1210). For example,security component 405 may use the user identifier, received in the request, to authenticate the user. Additionally, or alternatively,security component 405 may authenticatescheduler 220 or request information fromscheduler 220 to perform the authentication. If properly authenticated,security component 405 may pass the request toorchestrator component 415. If not properly authenticated,security component 405 may deny the request. -
Process 1200 may include identifying an interface and the orchestrator components for processing the request (block 1215). For example, orchestrator component 415 (e.g., master component 610) may receive the request (including the interface identifier) fromsecurity component 405.Master component 610 may, using the interface identifier from the request, query interface table 705, ofmetadata database 440, to determine the orchestrator components (e.g., one or more ofpreprocess component 620,initialization component 630,preparation component 640,process component 650, or completion component 660) to be invoked for the request. In addition,master component 610 may also identify, from the tables ofmetadata database 440, the rules that will be executed for the request, the servers and services that will be invoked for the request, and/or other information relating to the interface that will process the request. -
Process 1200 may further include processing the data using one or more ofpreprocess component 620,initialization component 630,preparation component 640,process component 650, or completion component 660 (block 1220). For example, if interface table 705 indicates thatpreprocess component 620 is to be invoked,preprocess component 620 may, based on the rules, receive the interface identifier and the file name frommaster component 610.Preprocess component 620 may accessmetadata database 440 to obtain, for example, information identifying the source device at which the bulk data is located (e.g., an address of the source device), information identifying the target device to which the bulk data is to be transferred (e.g., an address of the target device), and information identifying the location of the bulk data on the source device.Preprocess component 620 may use this information to retrieve the bulk data from the source device and store the bulk data at the target device. In one implementation,preprocess component 620 may invoke an SFTP service to retrieve the bulk data, which may return success or failure as a parameter tomaster component 610. - If interface table 705 indicates that
initialization component 630 is to be invoked,initialization component 630 may, based on the rules, maintain session logs for the processing of the request.Initialization component 630 may also determine the name of the tables ofmetadata database 440 that are needed for processing the bulk data. - If interface table 705 indicates that
preparation component 640 is to be invoked,preparation component 640 may, based on the rules, convert the data (e.g., from the file, a database, etc.) to a canonical XML format. The conversion may be based on information obtained frommetadata database 440 and include one or more ODI service calls, one or more adapter calls (such as a database call, an FTP call, etc.), one or more Web Service Definition Language (WSDL) calls, one or more Java calls, and/or other types of calls. - If interface table 705 indicates that
process component 650 is to be invoked,process component 650 may, based on the rules, perform a translation and/or transformation operation. For example,master component 610 may send the interface identifier and the process type to the translation/transformation process. Based on the data from one or more of the tables ofmetadata database 440, the translation/transformation process may call the following services (from services component 425): an ODI/stored procedure service (e.g., the translation/transformation process may call the ODI service (or any service that is implemented in batch mode) to process the data from the canonical table); an adapter service; and/or an application service for applications (e.g., PeopleSoft, SAP, etc.); etc. Additionally, or alternatively,process component 650 may perform an edits/validations operation. This operation may include causing any business rules/edits that may be required to be executed. In one implementation, these business rules/edits may exist across multiple applications, such as PeopleSoft, COBOL, ODI, etc. As an example,process component 650 may receive the interface identifier and process type frommaster component 610 and use this information to access one or more tables inmetadata database 440 to obtain the details of the application/module that holds the edits definition and details. The application/module may access the shared memory to get the data for which the edits are to be performed, execute the edits, and store the data back in the shared memory. - If interface table 705 indicates that
completion component 660 is to be invoked,completion component 660 may, based on the rules, provide status information tonotification engine 670 regarding the success and failure of the interface execution. If the interface execution is successful,completion component 660 may push the data into the appropriate target system and notify (e.g., via a success message)notification engine 670, which may, in turn, notify the appropriate user(s) via electronic mail (e-mail) or in another manner.Completion component 660 may further notifymaster component 610 of the successful execution of the interface. If, on the other hand, the interface execution fails,completion component 660 may notify (e.g., via a failure message)notification engine 670, which may, in turn, notify the appropriate user(s) via e-mail or in another manner.Completion component 660 may further notifymaster component 610 of the unsuccessful execution of the interface. -
Process 1200 may also include performing monitoring of the execution of the interface (block 1225). In one implementation, while the appropriate interface ofinterface gateway 310 is being executing to process the bulk data, real-time monitor 450 may track the status of the processing of the bulk data. For example, real-time monitor 450 may track whether the bulk data has been successfully processed by one or more components oforchestrator component 415, whether the bulk data has been unsuccessfully processed by one or more components oforchestrator component 415, whether the bulk data is currently being processed by one or more components oforchestrator component 415, etc. -
Process 1200 may include storing the monitored information in transactional database 445 (block 1230). For example, real-time monitor 450 (or another component or components of interface gateway 310) may store the status of processing of the bulk data intransactional database 445. In one implementation, real-time monitor 450 (or another component or components of interface gateway 310) may only store error information intransactional database 445. The error information may include information identifying the data in the bulk data (e.g., the particular record) that experienced the error, information identifying the component oforchestrator component 415 that was processing the data when the error occurred, information identifying the service that was processing the data when the error occurred, the date/time when the error occurred, and/or other information. -
FIG. 13 is a flow chart of an exemplary process for monitoring the execution of an interface. In one implementation, the processing ofFIG. 13 may be performed byportal 320. In another implementation, some or all of the processing described below may be performed by one or more components, including or excludingportal 320. -
Process 1300 may include authenticating the user (block 1310). For example, a user atuser device 210 may access portal 320. Portal 320 (e.g., administration component 840) may authenticate the user by, for example, requesting a user identifier and password from the user, receiving the user identifier and password from the user, and comparing the received user identifier and password to a stored list of user identifiers and passwords. If the received user identifier and password match a user identifier and password in the list of user identifiers and passwords, portal 320 may determine that the user has been properly authenticated and may provide the user with access toportal 320. Alternatively, if the received user identifier and password do not match a user identifier and password in the list of user identifiers and passwords, portal 320 may deny the user access toportal 320. Assume that portal 320 has properly authenticated the user. -
Process 1300 may also include receiving a request for process monitoring (block 1320). For example, a user atuser device 210 may causeuser device 210 to transmit a request to monitor the processing of data byinterface gateway 310 toportal 320. -
Process 1300 may provide one or more graphical user interfaces to the user that allow the user to monitor the processing of data by interface gateway 310 (block 1330). For example, portal 320 (e.g., monitoring and audit component 820) may provide one or more graphical user interfaces that provide the current status of data that is being processed byinterface gateway 310. The current status may include information identifying one or more components oforchestrator component 415 that are currently processing the data and/or other information relating to the current status of the data or past status of the data. The status information may further include error information. -
FIG. 14 is a flow chart of an exemplary process for performing error correction. In one implementation, the processing ofFIG. 14 may be performed byportal 320. In another implementation, some or all of the processing described below may be performed by one or more components, including or excludingportal 320. -
Process 1400 may include authenticating the user (block 1420). For example, a user atuser device 210 may access portal 320. Portal 320 (e.g., administration component 840) may authenticate the user by, for example, requesting a user identifier and password from the user, receiving the user identifier and password from the user, and comparing the received user identifier and password to a stored list of user identifiers and passwords. If the received user identifier and password match a user identifier and password in the list of user identifiers and passwords, portal 320 may determine that the user has been properly authenticated and may provide the user with access toportal 320. Alternatively, if the received user identifier and password do not match a user identifier and password in the list of user identifiers and passwords, portal 320 may deny the user access toportal 320. Assume that portal 320 has properly authenticated the user. -
Process 1400 may also include receiving a request for performing error correction (block 1420). For example, a user atuser device 210 may causeuser device 210 to transmit a request to perform error correction toportal 320. -
Process 1400 may provide one or more graphical user interfaces to the user to allow the user to perform error correction (block 1430). For example, portal 320 (e.g., error correction component 830) may dynamically connect totransactional database 445 to obtain the data from corresponding tables for that interface and instance.Error correction component 830 may provide one or more graphical user interfaces to the user to allow the user to correct errors that occurred, for example, wheninterface gateway 310 was operating in a bulk mode.Error correction component 830 may provide a mass correction feature and a smart fix feature. The mass correction feature may cause errors to be corrected that match selected criteria. The smart fix feature may remember all the errors that have been fixed previously and can apply those corrections to future executions of the same interface. The smart fix feature may allow, for example, for default values to be given to any particular column of a record that includes a null value, selected values to be replaced with another value, selected values to be truncated, and/or other operations.Error correction component 830 may further allow the user to generate a report that shows the errors that have occurred when executing a particular instance of an interface, cause the instance of the interface to be re-executed, and/or perform other operations. - Implementations described herein provide a generic (one size fits all) interface gateway that can be used to implement any type of interface for various kinds of systems, such as ERP systems (e.g., SAP, PeopleSoft, etc.), Business Warehouse systems, Legacy systems, web services, business-to-business services, etc. In addition, the generic interface gateway can handle single payload requests, as well as batch request, where the payload is very big. The generic interface gateway may include a metadata-driven orchestration component that acts as the heart of the interface gateway. Users may configure an interface for the interface gateway by configuring the metadata-driven orchestration component to invoke whatever types of services are needed for processing the users' data. The orchestration component may read the metadata for the given interface to be executed and orchestrate the services in the defined order. The orchestration component may also decide whether the services can be triggered in sequential or parallel mode.
- In addition, a portal may be provided that allows for end-to-end status monitoring of the execution of an interface. The portal may also allow for dynamic error correction through, for example, a mass correction feature and/or a smart fix feature.
- The foregoing description of implementations provides illustration, but is not intended to be exhaustive or to limit the implementations to the precise form disclosed. Accordingly, modifications to the implementations described herein may be possible.
- For example, while series of blocks have been described with regard to FIGS. 9 and 11-14, the order of the blocks may be modified in other embodiments. Further, non-dependent blocks may be performed in parallel.
- It will be apparent that embodiments, as described herein, may be implemented in many different forms of software, firmware, and hardware in the embodiments illustrated in the figures. The actual software code or specialized control hardware used to implement embodiments described herein is not limiting of the invention. Thus, the operation and behavior of the embodiments were described without reference to the specific software code—it being understood that software and control hardware may be designed to implement the embodiments based on the description herein.
- Further, certain portions, described above, may be implemented as a component that performs one or more functions. A component, as used herein, may include hardware, such as a processor, ASIC, or FPGA, or a combination of hardware and software (e.g., a processor executing software).
- Even though particular combinations of features are recited in the claims and/or disclosed in the specification, these combinations are not intended to limit the disclosure of the invention. In fact, many of these features may be combined in ways not specifically recited in the claims and/or disclosed in the specification.
- No element, act, or instruction used in the present application should be construed as critical or essential unless explicitly described as such. Also, as used herein, the article “a” is intended to include one or more items. Where only one item is intended, the term “one” or similar language is used. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise.
Claims (24)
1. A method comprising:
receiving, by an interface gateway, a batch data request, the interface gateway being associated with a plurality of interfaces, each interface, of the plurality of interfaces, providing conversion between a different set of source and target formats, each interface further being associated with a combination of components from a group of components that includes:
a first component to fetch data from a source location and store the fetched data in a particular target location,
a second component to create session logs relating to a processing of data at the interface gateway,
a third component to convert data to a canonical format,
a fourth component to at least one of translate data, transform data, or edit data, and
a fifth component to forward data to a target system and notify one or more users of a successful or unsuccessful processing of data by the interface gateway;
identifying, by the interface gateway and based on receiving the batch data request, one interface, of the plurality of interfaces, with which the batch data request is to be processed; and
processing, by the interface gateway, batch data, associated with the batch data request, using the identified one interface, the processing causing the batch data to be converted to the target format, the processing including invoking the combination of components with which the identified one interface is associated.
2. The method of claim 1 , further comprising:
authenticating, prior to identifying the one interface, the batch data request.
3. The method of claim 1 , where the batch data request includes an interface identifier that corresponds to the one interface and a file name for the batch data,
where the processing includes invoking the first component, and
where the invoking the first component includes:
receiving, by the first component, the interface identifier and the file name,
using, by the first component, the interface identifier to determine first information identifying a source device where the batch data is stored, second information identifying a target device where the batch data is to be stored, and third information identifying a location, in the target device, where the batch data is to be stored,
retrieving, by the first component, the bulk data from the source device using the first information, and
storing the bulk data at the location in the target device using the second information and the third information.
4. The method of claim 1 , where the batch data request includes an interface identifier that corresponds to the one interface,
where the processing includes invoking the third component, and
where the invoking the third component includes:
converting the bulk data to a canonical eXtensible Markup Language (XML) structure, the canonical XML structure being identified using the interface identifier.
5. The method of claim 1 , where the batch data request includes an interface identifier that corresponds to the one interface,
where the processing includes invoking the fourth component, and
where the invoking the fourth component includes:
performing at least one of translating the bulk data, transforming the bulk data, or editing the bulk data based on the interface identifier.
6. The method of claim 1 , further comprising:
monitoring the processing of the bulk data by the one interface to create monitored data, and
providing the monitored data to a user.
7. The method of claim 1 , further comprising:
detecting an error occurring during the processing of the bulk data;
logging the detected error; and
providing information relating to the detected error to a user.
8. The method of claim 7 , further comprising:
receiving a signal from the user, and
reprocessing data, in the bulk data that is associated with the error, using the one interface, the reprocessing being performed in response to receiving the signal.
9. The method of claim 1 , further comprising:
providing a portal that allows a user to configure the one interface, where the configuring the one interface includes:
receiving first information specifying the combination of components with which the interface is associated,
receiving second information identifying steps to be executed by the combination of components and an order in which the steps are to be executed,
receiving third information identifying services that are to be invoked for executing the identified steps,
receiving fourth information identifying servers on which the identified services are located, and
associating the first information, the second information, the third information, and the fourth information in a database.
10. A system comprising:
an interface gateway that includes:
a memory to store a plurality of tables, the plurality of tables defining different interfaces that are used to process data that are to be transferred between source systems and target systems, the plurality of tables including:
a first table to store parameters relating to an interface of the different interfaces, the parameters including an interface identifier,
a second table to store a plurality of rules that are used to implement the interface, where a first rule, of the plurality of rules, causes received data to be converted to a canonical structure, and where a second rule, of the plurality of rules, causes the canonical structure to be translated or transformed to a target format,
a third table to store information mapping, for at least one rule of the rules stored in the second table, the at least one rule to a server and a service,
a fourth table to store parameters for the server,
a fifth table to store parameters for the service, and
a sixth table to store parameters for the canonical structure; and
an orchestrator component to:
receive a request to process data, where the request includes the interface identifier and where the data is in a source format,
identify the interface using the interface identifier,
process the data using the interface to convert the data to the target format, where the target format is different than the source format, where when processing the data, the orchestrator component is to:
execute the rules, where, when executing the rules, the orchestrator component is to:
execute the first rule to cause the data to be converted to the canonical structure,
execute the second rule to cause the canonical structure to be translated or transformed into the target format, and
execute the at least one rule by invoking the service on the server.
11. The system of claim 10 , where the request includes a real-time data request, and
where the orchestrator component receives the real-time data request from a user device.
12. The system of claim 10 , where the request includes a batch data request, and
where the orchestrator component receives the batch data request from a scheduler.
13. The system of claim 10 , where the interface gateway further includes:
an interface specific service component that connects to an input of the orchestrator component, the interface specific service component to:
convert data, received in connection with a real-time data request, to a canonical eXtensible Markup Language (XML) structure.
14. The system of claim 10 , where the interface gateway further includes:
a services component to:
implement one or more services for processing data received at the interface gateway, the one or more services including at least one of a monitoring service, a masking engine service, a data service that allows connectivity to a database, a data error handling service, a correction and recycling service, a notification service, an error handling service, or a Secure File Transfer Protocol (SFTP) service.
15. The system of claim 10 , where the interface gateway further includes:
a services component to:
implement a plurality of different types of services for processing data received at the interface gateway, the plurality of services being implemented as at least two of an Oracle Data Integration (ODI) service, a SAP service, a Java Web Service, or a Unix shell script.
16. The system of claim 15 , where the interface gateway further includes:
a service adapter component that includes a group of adapters for the different types of services implemented by the services component.
17. The system of claim 10 , where the interface gateway further includes:
a transactional database to:
store information relating to errors that occurred while processing the data by the interface gateway.
18. The system of claim 10 , further comprising:
a portal to:
provide one or more graphical user interfaces to a user that allow the user to create a new interface for the interface gateway, modify an existing interface associated with the interface gateway, and delete an existing interface associated with the interface gateway.
19. The system of claim 10 , where the portal is further to at least one of:
provide one or more graphical user interfaces to the user to allow the user to perform real-time monitoring of a status of data being processed by the interface gateway, or
provide one or more graphical user interfaces to the user to allow the user to perform error correction.
20. A method comprising:
receiving a request at an interface gateway, the request including a first interface identifier, the interface gateway being associated with a plurality of interfaces, each interface, of the plurality of interfaces, being associated with metadata that defines the interface, the metadata including:
an interface identifier,
information identifying services to be executed for the interface and an order in which the identified services are to be executed, and
information identifying servers on which the identified services are implemented;
identifying, by the interface gateway and for the received request, one interface, of the plurality of interfaces, for processing the request based on the first interface identifier; and
processing, by interface gateway and using the one interface, the received request, the processing including:
executing the identified services on the identified servers according to the order, the executing causing data, associated with the received request, to be converted from a source format to a target format.
21. The method of claim 20 , where the plurality of interfaces are user-configurable.
22. The method of claim 20 , where the request includes a batch data processing request.
23. The method of claim 20 , further comprising:
monitoring the processing of the received request to obtain status information; and
providing, via a graphical user interface, the status information to a user associated with the one interface.
24. The method of claim 20 , further comprising:
monitoring the processing of the received request to identify errors;
providing, via a graphical user interface, the errors to a user associated with the one interface; and
allowing the user to dynamically correct the identified errors.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/636,441 US20100205475A1 (en) | 2009-02-11 | 2009-12-11 | Meta-data driven, service-oriented architecture (soa)-enabled, application independent interface gateway |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15172809P | 2009-02-11 | 2009-02-11 | |
US12/636,441 US20100205475A1 (en) | 2009-02-11 | 2009-12-11 | Meta-data driven, service-oriented architecture (soa)-enabled, application independent interface gateway |
Publications (1)
Publication Number | Publication Date |
---|---|
US20100205475A1 true US20100205475A1 (en) | 2010-08-12 |
Family
ID=42541234
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/625,229 Active 2030-10-07 US8156159B2 (en) | 2009-02-11 | 2009-11-24 | Data masking and unmasking of sensitive data |
US12/636,441 Abandoned US20100205475A1 (en) | 2009-02-11 | 2009-12-11 | Meta-data driven, service-oriented architecture (soa)-enabled, application independent interface gateway |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/625,229 Active 2030-10-07 US8156159B2 (en) | 2009-02-11 | 2009-11-24 | Data masking and unmasking of sensitive data |
Country Status (1)
Country | Link |
---|---|
US (2) | US8156159B2 (en) |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100275419A1 (en) * | 2009-04-30 | 2010-11-04 | Millus Christian A | Table cloth and skirt securing system |
US20110314341A1 (en) * | 2010-06-21 | 2011-12-22 | Salesforce.Com, Inc. | Method and systems for a dashboard testing framework in an online demand service environment |
US20120047162A1 (en) * | 2010-08-20 | 2012-02-23 | Jenzabar, Inc. | Method and System for Securing Academic ERP Database using Datasource Proxy |
US20140195818A1 (en) * | 2013-01-09 | 2014-07-10 | Thomson Licensing | Method and device for privacy respecting data processing |
US20160092476A1 (en) * | 2014-09-26 | 2016-03-31 | Oracle International Corporation | Declarative external data source importation, exportation, and metadata reflection utilizing http and hdfs protocols |
US20170017714A1 (en) * | 2013-05-10 | 2017-01-19 | Uberfan, Llc | Event-related media management system |
US9575749B1 (en) | 2015-12-17 | 2017-02-21 | Kersplody Corporation | Method and apparatus for execution of distributed workflow processes |
US9774568B2 (en) * | 2015-06-30 | 2017-09-26 | AO Kaspersky Lab | Computer security architecture and related computing method |
US20180101562A1 (en) * | 2016-10-12 | 2018-04-12 | Bank Of America Corporation | Metadata Validation Tool |
US10169077B1 (en) * | 2016-11-28 | 2019-01-01 | United Services Automobile Association (Usaa) | Systems, devices, and methods for mainframe data management |
US10210246B2 (en) | 2014-09-26 | 2019-02-19 | Oracle International Corporation | Techniques for similarity analysis and data enrichment using knowledge sources |
US20200099794A1 (en) * | 2016-12-13 | 2020-03-26 | Bullhead Innovations Ltd. | Data gate apparatus for integrating functionalities of an interface format into a plurality of services and method thereof |
US20200356548A1 (en) * | 2019-05-09 | 2020-11-12 | Mastercard International Incorporated | Methods and systems for facilitating message format discovery in online transaction processing |
US10885056B2 (en) | 2017-09-29 | 2021-01-05 | Oracle International Corporation | Data standardization techniques |
US10891272B2 (en) | 2014-09-26 | 2021-01-12 | Oracle International Corporation | Declarative language and visualization system for recommended data transformations and repairs |
US10924482B1 (en) * | 2014-12-18 | 2021-02-16 | Amazon Technologies, Inc. | Virtual service authorization |
US10936599B2 (en) | 2017-09-29 | 2021-03-02 | Oracle International Corporation | Adaptive recommendations |
US11016831B2 (en) * | 2014-11-11 | 2021-05-25 | Fair Isaac Corporation | System and method for linearizing messages from data sources for optimized high-performance processing in a stream processing system |
Families Citing this family (112)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7877398B2 (en) * | 2007-11-19 | 2011-01-25 | International Business Machines Corporation | Masking related sensitive data in groups |
US8412755B2 (en) * | 2009-06-23 | 2013-04-02 | Hewlett-Packard Development Company, L.P. | Permuting records in a database for leak detection and tracing |
US8862999B2 (en) | 2010-11-22 | 2014-10-14 | International Business Machines Corporation | Dynamic de-identification of data |
US9323948B2 (en) * | 2010-12-14 | 2016-04-26 | International Business Machines Corporation | De-identification of data |
US8983985B2 (en) | 2011-01-28 | 2015-03-17 | International Business Machines Corporation | Masking sensitive data of table columns retrieved from a database |
US8930381B2 (en) * | 2011-04-07 | 2015-01-06 | Infosys Limited | Methods and systems for runtime data anonymization |
US8856157B2 (en) * | 2011-08-23 | 2014-10-07 | Business Objects Software Limited | Automatic detection of columns to be obfuscated in database schemas |
US20140012833A1 (en) * | 2011-09-13 | 2014-01-09 | Hans-Christian Humprecht | Protection of data privacy in an enterprise system |
US8930410B2 (en) | 2011-10-03 | 2015-01-06 | International Business Machines Corporation | Query transformation for masking data within database objects |
GB2501281A (en) * | 2012-04-18 | 2013-10-23 | Ibm | Masking data in the results of a database query |
CN102929499A (en) * | 2012-09-17 | 2013-02-13 | 鸿富锦精密工业(深圳)有限公司 | Password textbox display content control system |
US9026562B2 (en) * | 2012-10-05 | 2015-05-05 | Hazeltree Fund Services, Inc. | Methods and systems for agnostic data storage |
US9575977B1 (en) * | 2012-10-29 | 2017-02-21 | John H. Bergman | Data management system |
US9171182B2 (en) * | 2012-10-31 | 2015-10-27 | Tata Consultancy Services Limited | Dynamic data masking |
US10121023B2 (en) | 2012-12-18 | 2018-11-06 | Oracle International Corporation | Unveil information on prompt |
US9465954B1 (en) * | 2013-03-15 | 2016-10-11 | Dataguise Inc. | Method and system for tracking masking of data |
US9460311B2 (en) | 2013-06-26 | 2016-10-04 | Sap Se | Method and system for on-the-fly anonymization on in-memory databases |
US20150040237A1 (en) * | 2013-08-05 | 2015-02-05 | Xerox Corporation | Systems and methods for interactive creation of privacy safe documents |
US10325099B2 (en) * | 2013-12-08 | 2019-06-18 | Microsoft Technology Licensing, Llc | Managing sensitive production data |
GB201321768D0 (en) | 2013-12-10 | 2014-01-22 | Ibm | Desktop redaction and masking |
US20150215330A1 (en) * | 2014-01-30 | 2015-07-30 | Shine Security Ltd. | Methods and systems of controlling distribution of personal data over network(s) |
US9807639B2 (en) * | 2014-01-30 | 2017-10-31 | Shine Security Ltd. | Network traffic event management at the client terminal level |
US10346374B1 (en) * | 2014-03-14 | 2019-07-09 | Open Invention Network Llc | Optimized data migration application for database compliant data extraction, loading and transformation |
US10789300B2 (en) * | 2014-04-28 | 2020-09-29 | Red Hat, Inc. | Method and system for providing security in a data federation system |
US9317558B2 (en) * | 2014-05-13 | 2016-04-19 | Sap Se | Intelligent unmasking in an in-memory database |
US20170220683A1 (en) * | 2014-07-31 | 2017-08-03 | Hewlett Packard Enterprise Development Lp | Shadow elements |
EP3198446A4 (en) | 2014-09-25 | 2018-04-04 | Hewlett-Packard Enterprise Development LP | A report comprising a masked value |
US10333899B2 (en) | 2014-11-26 | 2019-06-25 | Lexisnexis, A Division Of Reed Elsevier Inc. | Systems and methods for implementing a privacy firewall |
US9754027B2 (en) | 2014-12-12 | 2017-09-05 | International Business Machines Corporation | Implementation of data protection policies in ETL landscapes |
US9921976B2 (en) * | 2015-03-25 | 2018-03-20 | Vera | Access files |
US20170061155A1 (en) * | 2015-08-31 | 2017-03-02 | International Business Machines Corporation | Selective Policy Based Content Element Obfuscation |
US9922206B2 (en) * | 2015-10-02 | 2018-03-20 | Blackberry Limited | Private data exchange |
US9916465B1 (en) * | 2015-12-29 | 2018-03-13 | Palantir Technologies Inc. | Systems and methods for automatic and customizable data minimization of electronic data stores |
US9830149B2 (en) * | 2016-01-14 | 2017-11-28 | International Business Machines Corporation | Automatic extraction of sensitive code fragments to be executed in a sandbox |
US20220164840A1 (en) | 2016-04-01 | 2022-05-26 | OneTrust, LLC | Data processing systems and methods for integrating privacy information management systems with data loss prevention tools or other tools for privacy design |
US11461500B2 (en) | 2016-06-10 | 2022-10-04 | OneTrust, LLC | Data processing systems for cookie compliance testing with website scanning and related methods |
US11416590B2 (en) | 2016-06-10 | 2022-08-16 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US11354434B2 (en) | 2016-06-10 | 2022-06-07 | OneTrust, LLC | Data processing systems for verification of consent and notice processing and related methods |
US11354435B2 (en) | 2016-06-10 | 2022-06-07 | OneTrust, LLC | Data processing systems for data testing to confirm data deletion and related methods |
US10284604B2 (en) | 2016-06-10 | 2019-05-07 | OneTrust, LLC | Data processing and scanning systems for generating and populating a data inventory |
US10846433B2 (en) | 2016-06-10 | 2020-11-24 | OneTrust, LLC | Data processing consent management systems and related methods |
US11544667B2 (en) | 2016-06-10 | 2023-01-03 | OneTrust, LLC | Data processing systems for generating and populating a data inventory |
US11294939B2 (en) | 2016-06-10 | 2022-04-05 | OneTrust, LLC | Data processing systems and methods for automatically detecting and documenting privacy-related aspects of computer software |
US11366909B2 (en) | 2016-06-10 | 2022-06-21 | OneTrust, LLC | Data processing and scanning systems for assessing vendor risk |
US11416109B2 (en) | 2016-06-10 | 2022-08-16 | OneTrust, LLC | Automated data processing systems and methods for automatically processing data subject access requests using a chatbot |
US11403377B2 (en) | 2016-06-10 | 2022-08-02 | OneTrust, LLC | Privacy management systems and methods |
US10997318B2 (en) | 2016-06-10 | 2021-05-04 | OneTrust, LLC | Data processing systems for generating and populating a data inventory for processing data access requests |
US11675929B2 (en) | 2016-06-10 | 2023-06-13 | OneTrust, LLC | Data processing consent sharing systems and related methods |
US10678945B2 (en) | 2016-06-10 | 2020-06-09 | OneTrust, LLC | Consent receipt management systems and related methods |
US11562097B2 (en) | 2016-06-10 | 2023-01-24 | OneTrust, LLC | Data processing systems for central consent repository and related methods |
US11392720B2 (en) | 2016-06-10 | 2022-07-19 | OneTrust, LLC | Data processing systems for verification of consent and notice processing and related methods |
US11416798B2 (en) | 2016-06-10 | 2022-08-16 | OneTrust, LLC | Data processing systems and methods for providing training in a vendor procurement process |
US11222139B2 (en) | 2016-06-10 | 2022-01-11 | OneTrust, LLC | Data processing systems and methods for automatic discovery and assessment of mobile software development kits |
US11586700B2 (en) | 2016-06-10 | 2023-02-21 | OneTrust, LLC | Data processing systems and methods for automatically blocking the use of tracking tools |
US11481710B2 (en) | 2016-06-10 | 2022-10-25 | OneTrust, LLC | Privacy management systems and methods |
US11520928B2 (en) | 2016-06-10 | 2022-12-06 | OneTrust, LLC | Data processing systems for generating personal data receipts and related methods |
US11222142B2 (en) | 2016-06-10 | 2022-01-11 | OneTrust, LLC | Data processing systems for validating authorization for personal data collection, storage, and processing |
US11475136B2 (en) | 2016-06-10 | 2022-10-18 | OneTrust, LLC | Data processing systems for data transfer risk identification and related methods |
US11727141B2 (en) | 2016-06-10 | 2023-08-15 | OneTrust, LLC | Data processing systems and methods for synching privacy-related user consent across multiple computing devices |
US10318761B2 (en) | 2016-06-10 | 2019-06-11 | OneTrust, LLC | Data processing systems and methods for auditing data request compliance |
US11651106B2 (en) | 2016-06-10 | 2023-05-16 | OneTrust, LLC | Data processing systems for fulfilling data subject access requests and related methods |
US11410106B2 (en) | 2016-06-10 | 2022-08-09 | OneTrust, LLC | Privacy management systems and methods |
US11651104B2 (en) | 2016-06-10 | 2023-05-16 | OneTrust, LLC | Consent receipt management systems and related methods |
US11188615B2 (en) | 2016-06-10 | 2021-11-30 | OneTrust, LLC | Data processing consent capture systems and related methods |
US11636171B2 (en) | 2016-06-10 | 2023-04-25 | OneTrust, LLC | Data processing user interface monitoring systems and related methods |
US11438386B2 (en) | 2016-06-10 | 2022-09-06 | OneTrust, LLC | Data processing systems for data-transfer risk identification, cross-border visualization generation, and related methods |
US10740487B2 (en) | 2016-06-10 | 2020-08-11 | OneTrust, LLC | Data processing systems and methods for populating and maintaining a centralized database of personal data |
US11625502B2 (en) | 2016-06-10 | 2023-04-11 | OneTrust, LLC | Data processing systems for identifying and modifying processes that are subject to data subject access requests |
US11188862B2 (en) | 2016-06-10 | 2021-11-30 | OneTrust, LLC | Privacy management systems and methods |
US20180035285A1 (en) * | 2016-07-29 | 2018-02-01 | International Business Machines Corporation | Semantic Privacy Enforcement |
FR3061389B1 (en) * | 2016-12-22 | 2019-05-31 | Airbus Defence And Space Sas | SYSTEM AND METHOD FOR UNIDIRECTIONAL COMMUNICATION |
US10410014B2 (en) | 2017-03-23 | 2019-09-10 | Microsoft Technology Licensing, Llc | Configurable annotations for privacy-sensitive user content |
US10380355B2 (en) | 2017-03-23 | 2019-08-13 | Microsoft Technology Licensing, Llc | Obfuscation of user content in structured user data files |
US10671753B2 (en) | 2017-03-23 | 2020-06-02 | Microsoft Technology Licensing, Llc | Sensitive data loss protection for structured user content viewed in user applications |
US10013577B1 (en) | 2017-06-16 | 2018-07-03 | OneTrust, LLC | Data processing systems for identifying whether cookies contain personally identifying information |
US10909260B2 (en) | 2018-06-07 | 2021-02-02 | Jpmorgan Chase Bank, N.A. | Methods for data masking and devices thereof |
US11157563B2 (en) * | 2018-07-13 | 2021-10-26 | Bank Of America Corporation | System for monitoring lower level environment for unsanitized data |
US11100251B2 (en) | 2018-08-28 | 2021-08-24 | International Business Machines Corporation | Cleaning sensitive data from a diagnostic-ready clean copy |
US10891391B2 (en) | 2018-08-29 | 2021-01-12 | International Business Machines Corporation | Remote file storage with multiple access levels |
US10803202B2 (en) | 2018-09-07 | 2020-10-13 | OneTrust, LLC | Data processing systems for orphaned data identification and deletion and related methods |
US11544409B2 (en) | 2018-09-07 | 2023-01-03 | OneTrust, LLC | Data processing systems and methods for automatically protecting sensitive data within privacy management systems |
US10380380B1 (en) | 2018-10-08 | 2019-08-13 | Capital One Services, Llc | Protecting client personal data from customer service agents |
US11562085B2 (en) * | 2018-10-19 | 2023-01-24 | Oracle International Corporation | Anisotropic compression as applied to columnar storage formats |
WO2020082187A1 (en) * | 2018-10-26 | 2020-04-30 | Element Ai Inc. | Sensitive data detection and replacement |
US11227065B2 (en) * | 2018-11-06 | 2022-01-18 | Microsoft Technology Licensing, Llc | Static data masking |
US11100087B2 (en) | 2019-04-26 | 2021-08-24 | Microsoft Technology Licensing, Llc | Data tokenization system maintaining data integrity |
CN110348239B (en) * | 2019-06-13 | 2023-10-27 | 张建军 | Desensitization rule configuration method, data desensitization method, system and computer equipment |
US11507699B2 (en) * | 2019-09-27 | 2022-11-22 | Intel Corporation | Processor with private pipeline |
US11451371B2 (en) * | 2019-10-30 | 2022-09-20 | Dell Products L.P. | Data masking framework for information processing system |
EP4179435A1 (en) | 2020-07-08 | 2023-05-17 | OneTrust LLC | Systems and methods for targeted data discovery |
US20220012357A1 (en) * | 2020-07-10 | 2022-01-13 | Bank Of America Corporation | Intelligent privacy and security enforcement tool for unstructured data |
WO2022026564A1 (en) | 2020-07-28 | 2022-02-03 | OneTrust, LLC | Systems and methods for automatically blocking the use of tracking tools |
US11475165B2 (en) | 2020-08-06 | 2022-10-18 | OneTrust, LLC | Data processing systems and methods for automatically redacting unstructured data from a data subject access request |
US11308236B2 (en) | 2020-08-12 | 2022-04-19 | Kyndryl, Inc. | Managing obfuscation of regulated sensitive data |
US11436373B2 (en) | 2020-09-15 | 2022-09-06 | OneTrust, LLC | Data processing systems and methods for detecting tools for the automatic blocking of consent requests |
WO2022061270A1 (en) | 2020-09-21 | 2022-03-24 | OneTrust, LLC | Data processing systems and methods for automatically detecting target data transfers and target data processing |
WO2022099023A1 (en) | 2020-11-06 | 2022-05-12 | OneTrust, LLC | Systems and methods for identifying data processing activities based on data discovery results |
WO2022159901A1 (en) | 2021-01-25 | 2022-07-28 | OneTrust, LLC | Systems and methods for discovery, classification, and indexing of data in a native computing system |
US11442906B2 (en) | 2021-02-04 | 2022-09-13 | OneTrust, LLC | Managing custom attributes for domain objects defined within microservices |
WO2022170254A1 (en) * | 2021-02-08 | 2022-08-11 | OneTrust, LLC | Data processing systems and methods for anonymizing data samples in classification analysis |
US11601464B2 (en) | 2021-02-10 | 2023-03-07 | OneTrust, LLC | Systems and methods for mitigating risks of third-party computing system functionality integration into a first-party computing system |
WO2022178089A1 (en) | 2021-02-17 | 2022-08-25 | OneTrust, LLC | Managing custom workflows for domain objects defined within microservices |
WO2022178219A1 (en) | 2021-02-18 | 2022-08-25 | OneTrust, LLC | Selective redaction of media content |
US11687564B2 (en) * | 2021-02-24 | 2023-06-27 | Delphix Corp. | Continuous real-time masked database replication |
EP4305539A1 (en) | 2021-03-08 | 2024-01-17 | OneTrust, LLC | Data transfer discovery and analysis systems and related methods |
US11562078B2 (en) | 2021-04-16 | 2023-01-24 | OneTrust, LLC | Assessing and managing computational risk involved with integrating third party computing functionality within a computing system |
US11379617B1 (en) | 2021-10-22 | 2022-07-05 | Akoya LLC | Systems and methods for managing tokens and filtering data to control data access |
US11373000B1 (en) * | 2021-10-22 | 2022-06-28 | Akoya LLC | Systems and methods for managing tokens and filtering data to control data access |
US11379614B1 (en) | 2021-10-22 | 2022-07-05 | Akoya LLC | Systems and methods for managing tokens and filtering data to control data access |
US11641357B1 (en) | 2021-10-22 | 2023-05-02 | Akoya LLC | Systems and methods for managing tokens and filtering data to control data access |
US11496483B1 (en) | 2021-10-22 | 2022-11-08 | Akoya LLC | Systems and methods for managing tokens and filtering data to control data access |
US11620142B1 (en) | 2022-06-03 | 2023-04-04 | OneTrust, LLC | Generating and customizing user interfaces for demonstrating functions of interactive user environments |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6256676B1 (en) * | 1998-11-18 | 2001-07-03 | Saga Software, Inc. | Agent-adapter architecture for use in enterprise application integration systems |
US20060031584A1 (en) * | 2004-04-12 | 2006-02-09 | Mckinley James M | Web based dynamic data translation service and method |
US20070143334A1 (en) * | 2005-12-16 | 2007-06-21 | Microsoft Corporation | Electronic data interchange (EDI) schema simplification interface |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5014219A (en) * | 1988-05-06 | 1991-05-07 | White James A | Mask controled neural networks |
US6968390B1 (en) * | 1999-04-15 | 2005-11-22 | International Business Machines Corporation | Method and system for enabling a network function in a context of one or all server names in a multiple server name environment |
US6742034B1 (en) * | 1999-12-16 | 2004-05-25 | Dell Products L.P. | Method for storage device masking in a storage area network and storage controller and storage subsystem for using such a method |
US7020718B2 (en) * | 2000-05-15 | 2006-03-28 | Hewlett-Packard Development Company, L.P. | System and method of aggregating discontiguous address ranges into addresses and masks using a plurality of repeating address blocks |
US7228503B1 (en) * | 2000-08-25 | 2007-06-05 | Taiwan Semiconductor Manufacturing Co., Ltd. | Method for remote mask data jobview through a network |
US6647053B1 (en) * | 2000-08-31 | 2003-11-11 | Ricochet Networks, Inc. | Method and system for channel masking in a communication network |
US7425391B2 (en) * | 2001-10-02 | 2008-09-16 | Guobiao Zhang | Highly-corrected mask |
CA2690511C (en) * | 2003-03-13 | 2016-02-09 | 777388 Ontario Limited | Networked sound masking system with centralized sound masking generation |
FR2858896A1 (en) * | 2003-08-12 | 2005-02-18 | France Telecom | METHOD OF MASKING APPLICATION TREATMENTS OF SERVER ACCESS REQUEST AND CORRESPONDING MASKING SYSTEM |
US7281083B2 (en) * | 2004-06-30 | 2007-10-09 | Intel Corporation | Network processor with content addressable memory (CAM) mask |
US20070016637A1 (en) * | 2005-07-18 | 2007-01-18 | Brawn John M | Bitmap network masks |
FR2892585A1 (en) * | 2005-10-26 | 2007-04-27 | France Telecom | METHOD AND SYSTEM FOR PROTECTING A LINK OF ACCESS TO A SERVER. |
US8107639B2 (en) * | 2006-06-29 | 2012-01-31 | 777388 Ontario Limited | System and method for a sound masking system for networked workstations or offices |
FR2925997B1 (en) * | 2007-12-31 | 2010-02-12 | Radiotelephone Sfr | METHOD OF MASKING CELL IDENTIFIERS OR CODES OF LOCATION AREAS OF A MOBILE NETWORK RELATIVE TO A MOBILE TERMINAL |
US8208494B2 (en) * | 2008-12-03 | 2012-06-26 | Gigamon Llc | Intelligent packet slicing |
-
2009
- 2009-11-24 US US12/625,229 patent/US8156159B2/en active Active
- 2009-12-11 US US12/636,441 patent/US20100205475A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6256676B1 (en) * | 1998-11-18 | 2001-07-03 | Saga Software, Inc. | Agent-adapter architecture for use in enterprise application integration systems |
US20060031584A1 (en) * | 2004-04-12 | 2006-02-09 | Mckinley James M | Web based dynamic data translation service and method |
US20070143334A1 (en) * | 2005-12-16 | 2007-06-21 | Microsoft Corporation | Electronic data interchange (EDI) schema simplification interface |
Cited By (45)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100275419A1 (en) * | 2009-04-30 | 2010-11-04 | Millus Christian A | Table cloth and skirt securing system |
US9495282B2 (en) * | 2010-06-21 | 2016-11-15 | Salesforce.Com, Inc. | Method and systems for a dashboard testing framework in an online demand service environment |
US20110314341A1 (en) * | 2010-06-21 | 2011-12-22 | Salesforce.Com, Inc. | Method and systems for a dashboard testing framework in an online demand service environment |
US20120047162A1 (en) * | 2010-08-20 | 2012-02-23 | Jenzabar, Inc. | Method and System for Securing Academic ERP Database using Datasource Proxy |
US20140195818A1 (en) * | 2013-01-09 | 2014-07-10 | Thomson Licensing | Method and device for privacy respecting data processing |
US10740305B2 (en) | 2013-05-10 | 2020-08-11 | Uberfan, Llc | Event-related media management system |
US10176247B2 (en) | 2013-05-10 | 2019-01-08 | Uberfan, Llc | Event-related media management system |
US11899637B2 (en) | 2013-05-10 | 2024-02-13 | Uberfan, Llc | Event-related media management system |
US9727634B2 (en) | 2013-05-10 | 2017-08-08 | Uberfan, Llc | Event-related media management system |
US9754013B2 (en) * | 2013-05-10 | 2017-09-05 | Uberfan, Llc | Event-related media management system |
US11755551B2 (en) | 2013-05-10 | 2023-09-12 | Uberfan, Llc | Event-related media management system |
US9817883B2 (en) | 2013-05-10 | 2017-11-14 | Uberfan, Llc | Event-related media management system |
US10963439B1 (en) | 2013-05-10 | 2021-03-30 | Uberfan, Llc | Event-related media management system |
US20170017714A1 (en) * | 2013-05-10 | 2017-01-19 | Uberfan, Llc | Event-related media management system |
US20160092476A1 (en) * | 2014-09-26 | 2016-03-31 | Oracle International Corporation | Declarative external data source importation, exportation, and metadata reflection utilizing http and hdfs protocols |
US10210246B2 (en) | 2014-09-26 | 2019-02-19 | Oracle International Corporation | Techniques for similarity analysis and data enrichment using knowledge sources |
US10296192B2 (en) | 2014-09-26 | 2019-05-21 | Oracle International Corporation | Dynamic visual profiling and visualization of high volume datasets and real-time smart sampling and statistical profiling of extremely large datasets |
US11379506B2 (en) | 2014-09-26 | 2022-07-05 | Oracle International Corporation | Techniques for similarity analysis and data enrichment using knowledge sources |
US10976907B2 (en) * | 2014-09-26 | 2021-04-13 | Oracle International Corporation | Declarative external data source importation, exportation, and metadata reflection utilizing http and HDFS protocols |
US11693549B2 (en) * | 2014-09-26 | 2023-07-04 | Oracle International Corporation | Declarative external data source importation, exportation, and metadata reflection utilizing HTTP and HDFS protocols |
US10915233B2 (en) | 2014-09-26 | 2021-02-09 | Oracle International Corporation | Automated entity correlation and classification across heterogeneous datasets |
US10891272B2 (en) | 2014-09-26 | 2021-01-12 | Oracle International Corporation | Declarative language and visualization system for recommended data transformations and repairs |
US11016831B2 (en) * | 2014-11-11 | 2021-05-25 | Fair Isaac Corporation | System and method for linearizing messages from data sources for optimized high-performance processing in a stream processing system |
US11900181B2 (en) | 2014-11-11 | 2024-02-13 | Fair Isaac Corporation | System and method for linearizing messages from data sources for optimized high-performance processing in a stream processing system |
US10924482B1 (en) * | 2014-12-18 | 2021-02-16 | Amazon Technologies, Inc. | Virtual service authorization |
US9774568B2 (en) * | 2015-06-30 | 2017-09-26 | AO Kaspersky Lab | Computer security architecture and related computing method |
US10361998B2 (en) | 2015-06-30 | 2019-07-23 | AO Kaspersky Lab | Secure gateway communication systems and methods |
US10360024B2 (en) | 2015-12-17 | 2019-07-23 | Kersplody Corporation | Method and apparatus for execution of distributed workflow processes |
US9575749B1 (en) | 2015-12-17 | 2017-02-21 | Kersplody Corporation | Method and apparatus for execution of distributed workflow processes |
US20180101562A1 (en) * | 2016-10-12 | 2018-04-12 | Bank Of America Corporation | Metadata Validation Tool |
US11182375B2 (en) | 2016-10-12 | 2021-11-23 | Bank Of America Corporation | Metadata validation tool |
US10474666B2 (en) * | 2016-10-12 | 2019-11-12 | Bank Of America Corporation | Metadata validation tool |
US10521266B1 (en) | 2016-11-28 | 2019-12-31 | United Services Automobile Association (Usaa) | Systems, devices, and methods for mainframe data management |
US10831531B1 (en) | 2016-11-28 | 2020-11-10 | United Services Automobile Association (Usaa) | Systems, devices, and methods for mainframe data management |
US10169077B1 (en) * | 2016-11-28 | 2019-01-01 | United Services Automobile Association (Usaa) | Systems, devices, and methods for mainframe data management |
US11481246B1 (en) | 2016-11-28 | 2022-10-25 | United Services Automobile Association (Usaa) | Systems, devices, and methods for mainframe data management |
US11797335B1 (en) * | 2016-11-28 | 2023-10-24 | United Services Automobile Association (Usaa) | Systems, devices, and methods for mainframe data management |
US11196872B2 (en) | 2016-12-13 | 2021-12-07 | Bullhead Innovations Ltd. | System for voice control of devices at hospitality establishment and method and control server thereof |
US11652925B2 (en) | 2016-12-13 | 2023-05-16 | Bullhead Innovations Ltd. | Voice-controlled system that checks program guide to determine channel on which program name determined according to voice-to-text transcript is currently playing |
US20200099794A1 (en) * | 2016-12-13 | 2020-03-26 | Bullhead Innovations Ltd. | Data gate apparatus for integrating functionalities of an interface format into a plurality of services and method thereof |
US10887469B2 (en) * | 2016-12-13 | 2021-01-05 | Bullhead Innovations Ltd. | Data gate apparatus for integrating functionalities of an interface format into a plurality of services and method thereof |
US10936599B2 (en) | 2017-09-29 | 2021-03-02 | Oracle International Corporation | Adaptive recommendations |
US11500880B2 (en) | 2017-09-29 | 2022-11-15 | Oracle International Corporation | Adaptive recommendations |
US10885056B2 (en) | 2017-09-29 | 2021-01-05 | Oracle International Corporation | Data standardization techniques |
US20200356548A1 (en) * | 2019-05-09 | 2020-11-12 | Mastercard International Incorporated | Methods and systems for facilitating message format discovery in online transaction processing |
Also Published As
Publication number | Publication date |
---|---|
US8156159B2 (en) | 2012-04-10 |
US20100205189A1 (en) | 2010-08-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20100205475A1 (en) | Meta-data driven, service-oriented architecture (soa)-enabled, application independent interface gateway | |
US10715630B2 (en) | Common information model interoperability system | |
US11574070B2 (en) | Application specific schema extensions for a hierarchical data structure | |
US7516440B2 (en) | System and method for providing a java interface to an application view component | |
US8290947B2 (en) | Federated search | |
US8627345B2 (en) | Apparatus, system, and method for soap access to data source procedures | |
US8626778B2 (en) | System and method for conversion of JMS message data into database transactions for application to multiple heterogeneous databases | |
US10860550B1 (en) | Versioning schemas for hierarchical data structures | |
JP2021533448A (en) | Systems and methods to support SQL-based rich queries in hyperlegger fabric blockchain | |
US8103673B2 (en) | Systems and methods for provisioning content from multiple sources to a computing device | |
US20100287158A1 (en) | Information access using ontologies | |
US20090313256A1 (en) | Reuse of shared metadata across applications via url protocol | |
US8745088B2 (en) | System and method of performing risk analysis using a portal | |
US8656056B2 (en) | Web-enabled mainframe | |
US9515948B2 (en) | Techniques for generically accessing data | |
US11861029B2 (en) | Workflow execution state variables | |
US20200334244A1 (en) | Bidirectional mapping of hierarchical data to database object types | |
US11829814B2 (en) | Resolving data location for queries in a multi-system instance landscape | |
US20090183175A1 (en) | Systems and methods for receiving and sending messages about changes to data attributes | |
US9632837B2 (en) | Systems and methods for system consolidation | |
US10404710B2 (en) | Methods and apparatuses for providing improved directory services | |
US11775514B2 (en) | Computer system architecture and application for intercommunications in divergent database management systems | |
US20100198947A1 (en) | System and Method for Dynamically Processing Electronic Data Between Multiple Data Sources | |
US20140149540A1 (en) | Decentralized administration of access to target systems in identity management | |
US20070271229A1 (en) | System and method for data searching among multiple enterprise applications |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: VERIZON PATENT AND LICENSING, INC., NEW JERSEY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:EBRAHIMI, FARIBORZ;HASSAN, WALID;SINGH, SUMIT;AND OTHERS;SIGNING DATES FROM 20091208 TO 20091211;REEL/FRAME:023644/0028 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION |