US20110202854A1 - Metadata Capture for Screen Sharing - Google Patents

Metadata Capture for Screen Sharing Download PDF

Info

Publication number
US20110202854A1
US20110202854A1 US12/707,025 US70702510A US2011202854A1 US 20110202854 A1 US20110202854 A1 US 20110202854A1 US 70702510 A US70702510 A US 70702510A US 2011202854 A1 US2011202854 A1 US 2011202854A1
Authority
US
United States
Prior art keywords
metadata
screen
region
relevant
share
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/707,025
Inventor
Yuk Chan
Tin H. To
Andrew Tsang
Wei Zhou
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to US12/707,025 priority Critical patent/US20110202854A1/en
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHAN, YUK, TO, TIN H., TSANG, ANDREW, ZHOU, WEI
Priority to PCT/EP2011/051493 priority patent/WO2011101246A1/en
Priority to TW100104810A priority patent/TW201203097A/en
Publication of US20110202854A1 publication Critical patent/US20110202854A1/en
Priority to US13/453,009 priority patent/US9021367B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay

Definitions

  • This disclosure relates generally to the field of screen sharing during online communication.
  • Screen sharing may be used to communicate information from a user's (or sender) graphical user interface (GUI) screen in, for example, an online chat or web conference.
  • GUI graphical user interface
  • the screen share may comprise an image of the user's GUI screen that is sent to a receiver.
  • the GUI screen may comprise metadata, including but not limited to confidentiality information, audit trail information, text, table cell color, a uniform resource locator (URL) link, a search query, or a persistent session ID.
  • the metadata is not captured in the screen share image; for example, metadata such as selectable text or clickable URL links may be lost.
  • the metadata must be captured and shared separately.
  • Metadata may be highlighted and copied, table cell color may be extracted from HTML code, a search query may be copied from a search textbox, a URL link may be right clicked, and “Copy Link Location” selected, a screen capture of a bitmap may be extracted by pushing the printscreen key, or a session ID may be invisible, with no available extraction method.
  • a sender of a screen share may not be able to extract more than one of type of metadata at a time.
  • the receiver may determine some metadata via post processing and analysis of received screen share image, using, for example, optical character recognition (OCR) or lexicographic analysis.
  • OCR optical character recognition
  • An exemplary embodiment of a method for metadata capture for screen sharing of a graphical user interface (GUI) screen includes determining by a screen share sending module a region of the GUI screen to be shared; analyzing the GUI screen by a metadata capture module to determine metadata relevant to the region to be shared; and sending a screen share of the region comprising the relevant metadata to a receiver by the screen share sending module.
  • GUI graphical user interface
  • An exemplary embodiment of a system for metadata capture for screen sharing of a graphical user interface (GUI) screen includes a screen share sending module configured to determine a region of the GUI screen to be shared; a metadata capture module configured to analyze the GUI screen to determine metadata relevant to the region to be shared; and wherein the screen share sending module is further configured to send a screen share of the region comprising the relevant metadata to a receiver.
  • GUI graphical user interface
  • An exemplary embodiment of a computer program product comprising a computer readable storage medium containing computer code that, when executed by a computer, implements a method for metadata capture for screen sharing of a graphical user interface (GUI) screen, wherein the method includes determining by a screen share sending module a region of the GUI screen to be shared; analyzing the GUI screen by a metadata capture module to determine metadata relevant to the region to be shared; and sending a screen share of the region comprising the relevant metadata to a receiver by the screen share sending module.
  • GUI graphical user interface
  • FIG. 1 illustrates an embodiment of a system for metadata capture for screen sharing.
  • FIG. 2 illustrates an embodiment of a method for metadata capture for screen sharing.
  • FIG. 3 illustrates an embodiment of a GUI screen with a region selected for screen sharing.
  • FIG. 4 illustrates an embodiment of a sender interface for metadata capture for screen sharing.
  • FIG. 5 illustrates an embodiment of a receiver interface for metadata capture for screen sharing.
  • FIG. 6 illustrates an embodiment of a computer that may be used in conjunction with systems and methods for metadata capture for screen sharing.
  • Embodiments of systems and methods for metadata capture for screen sharing are provided, with exemplary embodiments being discussed below in detail.
  • Capture of metadata may be performed automatically in a screen share of a GUI screen, rather than through manual copying and pasting by the sender.
  • the sender may select a region within the GUI screen for which metadata is to be extracted, or the sender may select the entire GUI screen. Once a region of the screen is selected, object recognition technology is applied to the selected region, and to objects outside of the selected region that are relevant to the selected region, to recognize and capture the metadata.
  • Some or all of the captured metadata may be sent to a receiver along with the screen share, according to the sender's preferences.
  • the metadata may be rendered in the receiver's system as HTML, an image map, native object, or any other desired format.
  • the captured metadata is ready for use by the receiver, without the need for additional processing. Metadata may be captured across traditional boundaries, such as windows, frames, images or tables, and from outside of the designated metadata capture region.
  • the metadata may be captured and shared in a real-time, collaborative environment, to enhance and simplify collaboration and information sharing.
  • Metadata may include but is not limited to authorship information, confidentiality information, audit trail information, table cell color, a persistent session ID, text (including characteristics such as content, font size, or line spacing), window information such as title or whether the window is minimized/maximized, or any other application specific information, such as a search string URL for a browser application.
  • Shared metadata may be re-rendered in the most appropriate format with respect to the application from which the metadata is extracted. For example, captured metadata may be presented to appear as if it is on its native application.
  • Application specific support for application windows in the screen share may be provided through application specific plugins. For example, when sharing a Power Point® presentation or Microsoft Office® document, metadata such as author or last modified date may be shared.
  • a mechanism may be used that that introspects the HTML document described by a standard format such as document object model (DOM).
  • DOM document object model
  • FIG. 1 illustrates an embodiment of a system 100 for capturing metadata for screen sharing.
  • Sending computer system 101 comprises a GUI screen 102 .
  • a user may select some or all of GUI screen 102 for a screen share.
  • Screen share sending module 103 invokes metadata capture module 104 to identify the metadata present in the selected region of GUI screen 102 .
  • Screen share sending module 103 then applies the user's preferences to the identified metadata, and shares the screen share with the desired metadata with receiving computer system 106 via network 105 .
  • the screen share and metadata are displayed to the receiver via screen share receiving module 107 , which may comprise a chat or web conferencing application in some embodiments.
  • Metadata capture module 104 may implement any appropriate GUI object navigation technology to determine properties such as location, size, content, and visibility of objects in the GUI screen 102 . By determining each object's coordinates in the GUI screen, metadata capture module 104 may determine if an object is within the selected region, partially within the region, or relevant to the selected region. The metadata capture module 104 may then capture metadata from objects determined to be within or relevant to the selected region. Metadata capture module 104 may implement technology including but not limited to object map from the .NET framework, Rational Function Tester, Spy++, Public Application Programming Interface (API), Microsoft Foundation Class (MFC), or SpyLib in some embodiments.
  • API Public Application Programming Interface
  • MFC Microsoft Foundation Class
  • FIG. 2 illustrates an embodiment of a method 200 for capturing metadata for screen sharing.
  • FIG. 2 is discussed with reference to FIG. 1 .
  • a sender selects a region of a GUI screen 102 to be shared as a screen share.
  • the selected region may comprise a portion of GUI screen 102 , or all of GUI screen 102 .
  • the selected region is analyzed by metadata capture module 104 to identify any metadata present in the selected region, and in any region of the GUI screen 102 relevant to the selected region if the selected region is less than the entire GUI screen 102 .
  • Data not currently visible in the GUI screen 102 but that may be made visible by scrolling, may also be captured.
  • the sender's preferences regarding which metadata to share in the screen share are determined and applied to the identified metadata to determine the metadata to be shared by screen share sending module 103 .
  • the sender preferences may be entered by the user at the time of the screen share, by, for example, displaying a list of available metadata to the user, or the preferences may comprise a previously stored set of preferences.
  • the screen share is transmitted by screen share sending module 103 to screen share receiving module 107 on receiving computer system 106 along with the metadata to be shared that was determined in block 203 .
  • Metadata may be captured from any portion of the GUI screen 102 that is determined to be relevant to the region of the GUI screen that is selected in block 201 .
  • FIG. 3 shows an embodiment of a GUI screen 300 with a region 305 of the GUI screen 300 selected for metadata capturing.
  • GUI screen 300 comprises various objects, including icons 301 a - d and windows 302 - 304 .
  • the selected region 305 may be selected in any appropriate manner, for example, clicking and dragging a mouse to form a box on a computer screen, or by making a circular region with finger on a touch-screen device in some embodiments.
  • a selected region 305 may comprise any portion or all of GUI screen 300 .
  • selected region 305 includes portions of windows 302 and 304 .
  • any metadata regarding windows 302 and 304 may be captured, whether the metadata is actually located inside of region 305 or not, as any metadata regarding windows 302 and 304 is relevant to objects located inside region 305 .
  • Information regarding icons 301 a - d and window 303 may not be captured.
  • the sender may designate preferences regarding the type and amount of metadata from selected region 305 to share. This may be accomplished via a sender interface such as is shown in FIG. 4 .
  • FIG. 4 illustrates an embodiment of a sender interface 400 for capturing metadata.
  • Interface 400 may be launched when a sender selects a region of a GUI screen for metadata capture; however, interface 400 is shown for illustrative purposes only, metadata capture may be performed via any appropriate interface.
  • Interface 400 comprises MyCapture 401 , which corresponds to the selected region of the sender's GUI screen, such as selected region 305 of FIG. 3 .
  • MyCapture 401 comprises a web browser application having a URL 402 , a search string 403 , and various clickable links such as clickable link 404 .
  • MyCapture 401 also comprises applications MyDisclosures 405 and Infoprint Footprint Manager 406 .
  • Available metadata window 407 lists the metadata available in MyCapture 401 , and allows the sender to use checkboxes to designate which of the listed metadata to send in a screen share of MyCapture 401 .
  • the user may select “Share With Metadata” button 408 .
  • FIG. 5 illustrates an embodiment of a receiver interface 500 for receiving a screen share 501 comprising captured metadata.
  • Receiver interface 500 comprises a chat window; however, receiver interface 500 is shown for illustrative purposes only, a screen share 501 comprising metadata may be received in type of communication.
  • Screen share 501 corresponds to MyCapture 401 as is shown in FIG. 4 , and comprises selectable URL link 502 , copyable search string 503 , clickable link 502 , and information regarding applications 505 and 506 .
  • code segment shows an embodiment of computer code that may be embodied in metadata capture module 104 to capture all metadata in a window:
  • the following code segment shows an embodiment an embodiment of computer code that may be embodied in metadata capture module 104 to capture metadata for a button in a selected region:
  • metadata capture module 104 may implement any appropriate metadata recognition technology.
  • FIG. 6 illustrates an example of a computer 600 which may be utilized by exemplary embodiments of systems and methods for metadata capture for screen sharing as embodied in software.
  • Various operations discussed above may utilize the capabilities of the computer 600 .
  • One or more of the capabilities of the computer 600 may be incorporated in any element, module, application, and/or component discussed herein.
  • the computer 600 includes, but is not limited to, PCs, workstations, laptops, PDAs, palm devices, servers, storages, and the like.
  • the computer 600 may include one or more processors 610 , memory 620 , and one or more input and/or output (I/O) devices 670 that are communicatively coupled via a local interface (not shown).
  • the local interface can be, for example but not limited to, one or more buses or other wired or wireless connections, as is known in the art.
  • the local interface may have additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communications. Further, the local interface may include address, control, and/or data connections to enable appropriate communications among the aforementioned components.
  • the processor 610 is a hardware device for executing software that can be stored in the memory 620 .
  • the processor 610 can be virtually any custom made or commercially available processor, a central processing unit (CPU), a digital signal processor (DSP), or an auxiliary processor among several processors associated with the computer 600 , and the processor 610 may be a semiconductor based microprocessor (in the form of a microchip) or a macroprocessor.
  • the memory 620 can include any one or combination of volatile memory elements (e.g., random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), etc.) and nonvolatile memory elements (e.g., ROM, erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), tape, compact disc read only memory (CD-ROM), disk, diskette, cartridge, cassette or the like, etc.).
  • RAM random access memory
  • DRAM dynamic random access memory
  • SRAM static random access memory
  • nonvolatile memory elements e.g., ROM, erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), tape, compact disc read only memory (CD-ROM), disk, diskette, cartridge, cassette or the like, etc.
  • the memory 620 may incorporate electronic, magnetic, optical, and/or other types of storage
  • the software in the memory 620 may include one or more separate programs, each of which comprises an ordered listing of executable instructions for implementing logical functions.
  • the software in the memory 620 includes a suitable operating system (O/S) 650 , compiler 640 , source code 630 , and one or more applications 660 in accordance with exemplary embodiments.
  • O/S operating system
  • the application 660 comprises numerous functional components for implementing the features and operations of the exemplary embodiments.
  • the application 660 of the computer 600 may represent various applications, computational units, logic, functional units, processes, operations, virtual entities, and/or modules in accordance with exemplary embodiments, but the application 660 is not meant to be a limitation.
  • the operating system 650 controls the execution of other computer programs, and provides scheduling, input-output control, file and data management, memory management, and communication control and related services. It is contemplated by the inventors that the application 660 for implementing exemplary embodiments may be applicable on all commercially available operating systems.
  • Application 660 may be a source program, executable program (object code), script, or any other entity comprising a set of instructions to be performed.
  • a source program then the program is usually translated via a compiler (such as the compiler 640 ), assembler, interpreter, or the like, which may or may not be included within the memory 620 , so as to operate properly in connection with the O/S 650 .
  • the application 660 can be written as an object oriented programming language, which has classes of data and methods, or a procedure programming language, which has routines, subroutines, and/or functions, for example but not limited to, C, C++, C#, Pascal, BASIC, API calls, HTML, XHTML, XML, ASP scripts, FORTRAN, COBOL, Perl, Java, ADA, .NET, and the like.
  • the I/O devices 670 may include input devices such as, for example but not limited to, a mouse, keyboard, scanner, microphone, camera, etc. Furthermore, the I/O devices 670 may also include output devices, for example but not limited to a printer, display, etc. Finally, the I/O devices 670 may further include devices that communicate both inputs and outputs, for instance but not limited to, a NIC or modulator/demodulator (for accessing remote devices, other files, devices, systems, or a network), a radio frequency (RF) or other transceiver, a telephonic interface, a bridge, a router, etc. The I/O devices 670 also include components for communicating over various networks, such as the Internet or intranet.
  • a NIC or modulator/demodulator for accessing remote devices, other files, devices, systems, or a network
  • RF radio frequency
  • the I/O devices 670 also include components for communicating over various networks, such as the Internet or intranet.
  • the software in the memory 620 may further include a basic input output system (BIOS) (omitted for simplicity).
  • BIOS is a set of essential software routines that initialize and test hardware at startup, start the O/S 650 , and support the transfer of data among the hardware devices.
  • the BIOS is stored in some type of read-only-memory, such as ROM, PROM, EPROM, EEPROM or the like, so that the BIOS can be executed when the computer 600 is activated.
  • the processor 610 When the computer 600 is in operation, the processor 610 is configured to execute software stored within the memory 620 , to communicate data to and from the memory 620 , and to generally control operations of the computer 600 pursuant to the software.
  • the application 660 and the O/S 650 are read, in whole or in part, by the processor 610 , perhaps buffered within the processor 610 , and then executed.
  • a computer readable medium may be an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program for use by or in connection with a computer related system or method.
  • the application 660 can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
  • a “computer-readable medium” can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
  • the computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.
  • the computer-readable medium may include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic or optical), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc memory (CDROM, CD R/W) (optical).
  • the computer-readable medium could even be paper or another suitable medium, upon which the program is printed or punched, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
  • the application 660 can be implemented with any one or a combination of the following technologies, which are well known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon data signals, an application specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc.
  • ASIC application specific integrated circuit
  • PGA programmable gate array
  • FPGA field programmable gate array
  • the technical effects and benefits of exemplary embodiments include automatic sending of metadata information in a screen sharing application without interrupting real-time communication.

Abstract

A method for metadata capture for screen sharing of a graphical user interface (GUI) screen includes determining by a screen share sending module a region of the GUI screen to be shared; analyzing the GUI screen by a metadata capture module to determine metadata relevant to the region to be shared; and sending a screen share of the region comprising the relevant metadata to a receiver by the screen share sending module.

Description

    BACKGROUND
  • This disclosure relates generally to the field of screen sharing during online communication.
  • Screen sharing may be used to communicate information from a user's (or sender) graphical user interface (GUI) screen in, for example, an online chat or web conference. The screen share may comprise an image of the user's GUI screen that is sent to a receiver. The GUI screen may comprise metadata, including but not limited to confidentiality information, audit trail information, text, table cell color, a uniform resource locator (URL) link, a search query, or a persistent session ID. The metadata is not captured in the screen share image; for example, metadata such as selectable text or clickable URL links may be lost. The metadata must be captured and shared separately.
  • Different types of metadata require different capturing methods. For example, text may be highlighted and copied, table cell color may be extracted from HTML code, a search query may be copied from a search textbox, a URL link may be right clicked, and “Copy Link Location” selected, a screen capture of a bitmap may be extracted by pushing the printscreen key, or a session ID may be invisible, with no available extraction method. A sender of a screen share may not be able to extract more than one of type of metadata at a time. The receiver may determine some metadata via post processing and analysis of received screen share image, using, for example, optical character recognition (OCR) or lexicographic analysis. However, such analysis does not capture all types of metadata that are present in the sender's GUI screen at the time of the screen share, and may perform inefficiently or inaccurately. Therefore, in an instant messaging scenario, the sender may need to copy and paste the metadata information separately and manually to the receiving party, which limits the usefulness of the screen share. In a web conference scenario, the meeting host may not be able to copy and paste metadata without interrupting the realtime nature of the web conference.
  • SUMMARY
  • An exemplary embodiment of a method for metadata capture for screen sharing of a graphical user interface (GUI) screen includes determining by a screen share sending module a region of the GUI screen to be shared; analyzing the GUI screen by a metadata capture module to determine metadata relevant to the region to be shared; and sending a screen share of the region comprising the relevant metadata to a receiver by the screen share sending module.
  • An exemplary embodiment of a system for metadata capture for screen sharing of a graphical user interface (GUI) screen includes a screen share sending module configured to determine a region of the GUI screen to be shared; a metadata capture module configured to analyze the GUI screen to determine metadata relevant to the region to be shared; and wherein the screen share sending module is further configured to send a screen share of the region comprising the relevant metadata to a receiver.
  • An exemplary embodiment of a computer program product comprising a computer readable storage medium containing computer code that, when executed by a computer, implements a method for metadata capture for screen sharing of a graphical user interface (GUI) screen, wherein the method includes determining by a screen share sending module a region of the GUI screen to be shared; analyzing the GUI screen by a metadata capture module to determine metadata relevant to the region to be shared; and sending a screen share of the region comprising the relevant metadata to a receiver by the screen share sending module.
  • Additional features are realized through the techniques of the present exemplary embodiment. Other embodiments are described in detail herein and are considered a part of what is claimed. For a better understanding of the features of the exemplary embodiment, refer to the description and to the drawings.
  • BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
  • Referring now to the drawings wherein like elements are numbered alike in the several FIGURES:
  • FIG. 1 illustrates an embodiment of a system for metadata capture for screen sharing.
  • FIG. 2 illustrates an embodiment of a method for metadata capture for screen sharing.
  • FIG. 3 illustrates an embodiment of a GUI screen with a region selected for screen sharing.
  • FIG. 4 illustrates an embodiment of a sender interface for metadata capture for screen sharing.
  • FIG. 5 illustrates an embodiment of a receiver interface for metadata capture for screen sharing.
  • FIG. 6 illustrates an embodiment of a computer that may be used in conjunction with systems and methods for metadata capture for screen sharing.
  • DETAILED DESCRIPTION
  • Embodiments of systems and methods for metadata capture for screen sharing are provided, with exemplary embodiments being discussed below in detail.
  • Capture of metadata may be performed automatically in a screen share of a GUI screen, rather than through manual copying and pasting by the sender. The sender may select a region within the GUI screen for which metadata is to be extracted, or the sender may select the entire GUI screen. Once a region of the screen is selected, object recognition technology is applied to the selected region, and to objects outside of the selected region that are relevant to the selected region, to recognize and capture the metadata. Some or all of the captured metadata may be sent to a receiver along with the screen share, according to the sender's preferences. The metadata may be rendered in the receiver's system as HTML, an image map, native object, or any other desired format. The captured metadata is ready for use by the receiver, without the need for additional processing. Metadata may be captured across traditional boundaries, such as windows, frames, images or tables, and from outside of the designated metadata capture region. The metadata may be captured and shared in a real-time, collaborative environment, to enhance and simplify collaboration and information sharing.
  • Metadata may include but is not limited to authorship information, confidentiality information, audit trail information, table cell color, a persistent session ID, text (including characteristics such as content, font size, or line spacing), window information such as title or whether the window is minimized/maximized, or any other application specific information, such as a search string URL for a browser application. Shared metadata may be re-rendered in the most appropriate format with respect to the application from which the metadata is extracted. For example, captured metadata may be presented to appear as if it is on its native application. Application specific support for application windows in the screen share may be provided through application specific plugins. For example, when sharing a Power Point® presentation or Microsoft Office® document, metadata such as author or last modified date may be shared. When sharing a browser window, a mechanism may be used that that introspects the HTML document described by a standard format such as document object model (DOM). When sharing a webpage with persistence and shareable sessions, the URL to access the session may also be sent as metadata.
  • FIG. 1 illustrates an embodiment of a system 100 for capturing metadata for screen sharing. Sending computer system 101 comprises a GUI screen 102. A user may select some or all of GUI screen 102 for a screen share. Screen share sending module 103 invokes metadata capture module 104 to identify the metadata present in the selected region of GUI screen 102. Screen share sending module 103 then applies the user's preferences to the identified metadata, and shares the screen share with the desired metadata with receiving computer system 106 via network 105. The screen share and metadata are displayed to the receiver via screen share receiving module 107, which may comprise a chat or web conferencing application in some embodiments.
  • Metadata capture module 104 may implement any appropriate GUI object navigation technology to determine properties such as location, size, content, and visibility of objects in the GUI screen 102. By determining each object's coordinates in the GUI screen, metadata capture module 104 may determine if an object is within the selected region, partially within the region, or relevant to the selected region. The metadata capture module 104 may then capture metadata from objects determined to be within or relevant to the selected region. Metadata capture module 104 may implement technology including but not limited to object map from the .NET framework, Rational Function Tester, Spy++, Public Application Programming Interface (API), Microsoft Foundation Class (MFC), or SpyLib in some embodiments.
  • FIG. 2 illustrates an embodiment of a method 200 for capturing metadata for screen sharing. FIG. 2 is discussed with reference to FIG. 1. In block 201, a sender selects a region of a GUI screen 102 to be shared as a screen share. The selected region may comprise a portion of GUI screen 102, or all of GUI screen 102. In block 202, the selected region is analyzed by metadata capture module 104 to identify any metadata present in the selected region, and in any region of the GUI screen 102 relevant to the selected region if the selected region is less than the entire GUI screen 102. Data not currently visible in the GUI screen 102, but that may be made visible by scrolling, may also be captured. In block 203, the sender's preferences regarding which metadata to share in the screen share are determined and applied to the identified metadata to determine the metadata to be shared by screen share sending module 103. The sender preferences may be entered by the user at the time of the screen share, by, for example, displaying a list of available metadata to the user, or the preferences may comprise a previously stored set of preferences. In block 204, the screen share is transmitted by screen share sending module 103 to screen share receiving module 107 on receiving computer system 106 along with the metadata to be shared that was determined in block 203.
  • Metadata may be captured from any portion of the GUI screen 102 that is determined to be relevant to the region of the GUI screen that is selected in block 201. FIG. 3 shows an embodiment of a GUI screen 300 with a region 305 of the GUI screen 300 selected for metadata capturing. GUI screen 300 comprises various objects, including icons 301 a-d and windows 302-304. The selected region 305 may be selected in any appropriate manner, for example, clicking and dragging a mouse to form a box on a computer screen, or by making a circular region with finger on a touch-screen device in some embodiments. A selected region 305 may comprise any portion or all of GUI screen 300. In FIG. 3, selected region 305 includes portions of windows 302 and 304. Therefore, in a metadata capture for a screen share of region 305, any metadata regarding windows 302 and 304 may be captured, whether the metadata is actually located inside of region 305 or not, as any metadata regarding windows 302 and 304 is relevant to objects located inside region 305. Information regarding icons 301 a-d and window 303, however, may not be captured.
  • The sender may designate preferences regarding the type and amount of metadata from selected region 305 to share. This may be accomplished via a sender interface such as is shown in FIG. 4. FIG. 4 illustrates an embodiment of a sender interface 400 for capturing metadata. Interface 400 may be launched when a sender selects a region of a GUI screen for metadata capture; however, interface 400 is shown for illustrative purposes only, metadata capture may be performed via any appropriate interface. Interface 400 comprises MyCapture 401, which corresponds to the selected region of the sender's GUI screen, such as selected region 305 of FIG. 3. MyCapture 401 comprises a web browser application having a URL 402, a search string 403, and various clickable links such as clickable link 404. MyCapture 401 also comprises applications MyDisclosures 405 and Infoprint Footprint Manager 406. Available metadata window 407 lists the metadata available in MyCapture 401, and allows the sender to use checkboxes to designate which of the listed metadata to send in a screen share of MyCapture 401. Once the user has selected the desired metadata for sending with the screen share, the user may select “Share With Metadata” button 408.
  • FIG. 5 illustrates an embodiment of a receiver interface 500 for receiving a screen share 501 comprising captured metadata. Receiver interface 500 comprises a chat window; however, receiver interface 500 is shown for illustrative purposes only, a screen share 501 comprising metadata may be received in type of communication. Screen share 501 corresponds to MyCapture 401 as is shown in FIG. 4, and comprises selectable URL link 502, copyable search string 503, clickable link 502, and information regarding applications 505 and 506.
  • Referring again to FIG. 1, the following code segment shows an embodiment of computer code that may be embodied in metadata capture module 104 to capture all metadata in a window:
  • Windows using the ManagedSpyLib (available for free from MSDN):
    using System;
    using System.Text;
    using System.Diagnostics;
    using System.Windows.Forms;
    using System.Collections.Generic;
    using Microsoft.ManagedSpy;
    class Program
    {
     static void Main(string[ ] args)
     {
      Dictionary<int, int> topWindowCounts = new Dictionary<int, int>( );
      foreach (ControlProxy proxy in ControlProxy.TopLevelWindows)
      {
       if (!topWindowCounts.ContainsKey(proxy.OwningProcess.Id))
        {
         topWindowCounts.Add(proxy.OwningProcess.Id, 0);
        }
        topWindowCounts[proxy.OwningProcess.Id]++;
       }
      foreach (int pid in top WindowCounts.Keys)
      {
       Process p = Process.GetProcessById(pid);
       Console.WriteLine(“Process:”+p.ProcessName+“has”+
        topWindowCount[pid].TString( ) + “top level windows”);
      }
     }
    }
  • The following code segment shows an embodiment an embodiment of computer code that may be embodied in metadata capture module 104 to capture metadata for a button in a selected region:
  • foreach (Process p in
    Process.GetProcessesByName(“WindowsApplication1”))
    {
      if (p.Id != Process.GetCurrentProcess( ).Id)
      {
       ControlProxy proxy =
       ControlProxy.FromHandle(p.MainWindowHandle);
      string val = (string)proxy.GetValue(“MyStringValue”);
      .....
      }
    }
  • The above code segments are shown for illustrative purposes only; metadata capture module 104 may implement any appropriate metadata recognition technology.
  • FIG. 6 illustrates an example of a computer 600 which may be utilized by exemplary embodiments of systems and methods for metadata capture for screen sharing as embodied in software. Various operations discussed above may utilize the capabilities of the computer 600. One or more of the capabilities of the computer 600 may be incorporated in any element, module, application, and/or component discussed herein.
  • The computer 600 includes, but is not limited to, PCs, workstations, laptops, PDAs, palm devices, servers, storages, and the like. Generally, in terms of hardware architecture, the computer 600 may include one or more processors 610, memory 620, and one or more input and/or output (I/O) devices 670 that are communicatively coupled via a local interface (not shown). The local interface can be, for example but not limited to, one or more buses or other wired or wireless connections, as is known in the art. The local interface may have additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communications. Further, the local interface may include address, control, and/or data connections to enable appropriate communications among the aforementioned components.
  • The processor 610 is a hardware device for executing software that can be stored in the memory 620. The processor 610 can be virtually any custom made or commercially available processor, a central processing unit (CPU), a digital signal processor (DSP), or an auxiliary processor among several processors associated with the computer 600, and the processor 610 may be a semiconductor based microprocessor (in the form of a microchip) or a macroprocessor.
  • The memory 620 can include any one or combination of volatile memory elements (e.g., random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), etc.) and nonvolatile memory elements (e.g., ROM, erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), tape, compact disc read only memory (CD-ROM), disk, diskette, cartridge, cassette or the like, etc.). Moreover, the memory 620 may incorporate electronic, magnetic, optical, and/or other types of storage media. Note that the memory 620 can have a distributed architecture, where various components are situated remote from one another, but can be accessed by the processor 610.
  • The software in the memory 620 may include one or more separate programs, each of which comprises an ordered listing of executable instructions for implementing logical functions. The software in the memory 620 includes a suitable operating system (O/S) 650, compiler 640, source code 630, and one or more applications 660 in accordance with exemplary embodiments. As illustrated, the application 660 comprises numerous functional components for implementing the features and operations of the exemplary embodiments. The application 660 of the computer 600 may represent various applications, computational units, logic, functional units, processes, operations, virtual entities, and/or modules in accordance with exemplary embodiments, but the application 660 is not meant to be a limitation.
  • The operating system 650 controls the execution of other computer programs, and provides scheduling, input-output control, file and data management, memory management, and communication control and related services. It is contemplated by the inventors that the application 660 for implementing exemplary embodiments may be applicable on all commercially available operating systems.
  • Application 660 may be a source program, executable program (object code), script, or any other entity comprising a set of instructions to be performed. When a source program, then the program is usually translated via a compiler (such as the compiler 640), assembler, interpreter, or the like, which may or may not be included within the memory 620, so as to operate properly in connection with the O/S 650. Furthermore, the application 660 can be written as an object oriented programming language, which has classes of data and methods, or a procedure programming language, which has routines, subroutines, and/or functions, for example but not limited to, C, C++, C#, Pascal, BASIC, API calls, HTML, XHTML, XML, ASP scripts, FORTRAN, COBOL, Perl, Java, ADA, .NET, and the like.
  • The I/O devices 670 may include input devices such as, for example but not limited to, a mouse, keyboard, scanner, microphone, camera, etc. Furthermore, the I/O devices 670 may also include output devices, for example but not limited to a printer, display, etc. Finally, the I/O devices 670 may further include devices that communicate both inputs and outputs, for instance but not limited to, a NIC or modulator/demodulator (for accessing remote devices, other files, devices, systems, or a network), a radio frequency (RF) or other transceiver, a telephonic interface, a bridge, a router, etc. The I/O devices 670 also include components for communicating over various networks, such as the Internet or intranet.
  • If the computer 600 is a PC, workstation, intelligent device or the like, the software in the memory 620 may further include a basic input output system (BIOS) (omitted for simplicity). The BIOS is a set of essential software routines that initialize and test hardware at startup, start the O/S 650, and support the transfer of data among the hardware devices. The BIOS is stored in some type of read-only-memory, such as ROM, PROM, EPROM, EEPROM or the like, so that the BIOS can be executed when the computer 600 is activated.
  • When the computer 600 is in operation, the processor 610 is configured to execute software stored within the memory 620, to communicate data to and from the memory 620, and to generally control operations of the computer 600 pursuant to the software. The application 660 and the O/S 650 are read, in whole or in part, by the processor 610, perhaps buffered within the processor 610, and then executed.
  • When the application 660 is implemented in software it should be noted that the application 660 can be stored on virtually any computer readable medium for use by or in connection with any computer related system or method. In the context of this document, a computer readable medium may be an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program for use by or in connection with a computer related system or method.
  • The application 660 can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a “computer-readable medium” can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.
  • More specific examples (a nonexhaustive list) of the computer-readable medium may include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic or optical), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc memory (CDROM, CD R/W) (optical). Note that the computer-readable medium could even be paper or another suitable medium, upon which the program is printed or punched, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
  • In exemplary embodiments, where the application 660 is implemented in hardware, the application 660 can be implemented with any one or a combination of the following technologies, which are well known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon data signals, an application specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc.
  • The technical effects and benefits of exemplary embodiments include automatic sending of metadata information in a screen sharing application without interrupting real-time communication.
  • The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
  • The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.

Claims (20)

1. A method for metadata capture for screen sharing of a graphical user interface (GUI) screen, the method comprising:
determining by a screen share sending module a region of the GUI screen to be shared;
analyzing the GUI screen by a metadata capture module to determine metadata relevant to the region to be shared; and
sending a screen share of the region comprising the relevant metadata to a receiver by the screen share sending module.
2. The method of claim 1, wherein the metadata relevant to the region is located outside of the region.
3. The method of claim 2, wherein the metadata relevant to the region is located outside of a visible region of the GUI screen.
4. The method of claim 1, further comprising receiving the screen share of the region comprising the relevant metadata by a screen share receiving module, wherein the received screen share comprises the relevant metadata in the same format in which it is presented in the GUI screen.
5. The method of claim 1, further comprising displaying a list of the relevant metadata to a sender of the screen share, receiving a selection of a piece of the relevant metadata from the sender, and sending the screen share of the region with selected metadata.
6. The method of claim 1, further comprising applying a set of stored preferences to the relevant metadata, and sending the screen share of the region with metadata designated by the set of stored preferences.
7. The method of claim 1, wherein the relevant metadata comprises authorship of a document, confidentiality information, audit trail information, table cell color, a persistent session ID, text content, text font size, text line spacing, window title, whether a window is minimized or maximized, or a search string.
8. The method of claim 1, wherein the screen share is sent as part of a web conference or chat session.
9. A system for metadata capture for screen sharing of a graphical user interface (GUI) screen, the system comprising:
a screen share sending module configured to determine a region of the GUI screen to be shared;
a metadata capture module configured to analyze the GUI screen to determine metadata relevant to the region to be shared; and
wherein the screen share sending module is further configured to send a screen share of the region comprising the relevant metadata to a receiver.
10. The system of claim 9, wherein the metadata relevant to the region is located outside of the region.
11. The system of claim 10, wherein the metadata relevant to the region is located outside of a visible region of the GUI screen.
12. The system of claim 9, further comprising a screen share receiving module configured to receive the screen share of the region comprising the relevant metadata, wherein the received screen share comprises the relevant metadata in the same format in which it is presented in the GUI screen.
13. The system of claim 9, wherein the screen share sending module is further configured to display a list of the relevant metadata to a sender of the screen share, receive a selection of a piece of the relevant metadata from the sender, and send the screen share of the region with selected metadata.
14. The system of claim 9, wherein the screen share sending module is further configured to apply a set of stored preferences to the relevant metadata, and send the screen share of the region with metadata designated by the set of stored preferences.
15. The system of claim 9, wherein the relevant metadata comprises authorship of a document, confidentiality information, audit trail information, table cell color, a persistent session ID, text content, text font size, text line spacing, window title, whether a window is minimized or maximized, or a search string.
16. The system of claim 8, wherein the screen share is sent as part of a web conference or chat session.
17. A computer program product comprising a computer readable storage medium containing computer code that, when executed by a computer, implements a method for metadata capture for screen sharing of a graphical user interface (GUI) screen, wherein the method comprises:
determining by a screen share sending module a region of the GUI screen to be shared;
analyzing the GUI screen by a metadata capture module to determine metadata relevant to the region to be shared; and
sending a screen share of the region comprising the relevant metadata to a receiver by the screen share sending module.
18. The computer program product according to claim 15, wherein the metadata relevant to the region is located outside of the region.
19. The computer program product according to claim 15, further comprising receiving the screen share of the region comprising the relevant metadata by a screen share receiving module, wherein the received screen share comprises the relevant metadata in the same format in which it is presented in the GUI screen.
20. The computer program product according to claim 15, further comprising displaying a list of the relevant metadata to a sender of the screen share, receiving a selection of a piece of the relevant metadata from the sender, and sending the screen share of the region with selected metadata.
US12/707,025 2010-02-17 2010-02-17 Metadata Capture for Screen Sharing Abandoned US20110202854A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US12/707,025 US20110202854A1 (en) 2010-02-17 2010-02-17 Metadata Capture for Screen Sharing
PCT/EP2011/051493 WO2011101246A1 (en) 2010-02-17 2011-02-02 Metadata capture for screen sharing
TW100104810A TW201203097A (en) 2010-02-17 2011-02-14 Metadata capture for screen sharing
US13/453,009 US9021367B2 (en) 2010-02-17 2012-04-23 Metadata capture for screen sharing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US12/707,025 US20110202854A1 (en) 2010-02-17 2010-02-17 Metadata Capture for Screen Sharing

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/453,009 Continuation US9021367B2 (en) 2010-02-17 2012-04-23 Metadata capture for screen sharing

Publications (1)

Publication Number Publication Date
US20110202854A1 true US20110202854A1 (en) 2011-08-18

Family

ID=43630022

Family Applications (2)

Application Number Title Priority Date Filing Date
US12/707,025 Abandoned US20110202854A1 (en) 2010-02-17 2010-02-17 Metadata Capture for Screen Sharing
US13/453,009 Expired - Fee Related US9021367B2 (en) 2010-02-17 2012-04-23 Metadata capture for screen sharing

Family Applications After (1)

Application Number Title Priority Date Filing Date
US13/453,009 Expired - Fee Related US9021367B2 (en) 2010-02-17 2012-04-23 Metadata capture for screen sharing

Country Status (3)

Country Link
US (2) US20110202854A1 (en)
TW (1) TW201203097A (en)
WO (1) WO2011101246A1 (en)

Cited By (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120317500A1 (en) * 2011-06-07 2012-12-13 At&T Intellectual Property I, L.P. System and method for data visualization and user collaboration
US20130111360A1 (en) * 2011-10-28 2013-05-02 Justin Kodama Accessed Location of User Interface
US20130212490A1 (en) * 2012-02-09 2013-08-15 International Business Machines Corporation Augmented screen sharing in an electronic meeting
US20130227455A1 (en) * 2012-02-24 2013-08-29 Samsung Electronics Co. Ltd. Method of sharing content and mobile terminal thereof
US20130239020A1 (en) * 2012-03-12 2013-09-12 Samsung Electronics Co., Ltd. Electronic-book system and method for sharing additional page information thereof
US20140032698A1 (en) * 2012-07-24 2014-01-30 Cisco Technology, Inc. Utilize Extra Web Semantic for Video Caching
WO2014084771A1 (en) * 2012-11-30 2014-06-05 Vergic Group Ab Dynamic sharing and updating of a web page
WO2014113684A1 (en) * 2013-01-17 2014-07-24 Futurewei Technologies, Inc. System, apparatus and method for sharing a screen having multiple visual components
US20140310599A1 (en) * 2013-04-12 2014-10-16 Sony Corporation Automatic discovery and mirroring of server-client remote user interface (rui) session on a companion device and synchronously controlling both sessions using rui on companion device
US20140331145A1 (en) * 2013-05-06 2014-11-06 Nvidia Corporation Enhancing a remote desktop with meta-information
US20150067106A1 (en) * 2013-08-28 2015-03-05 Mersive Technologies, Inc. Graphical Indicator Of Presence, Identity, And Action For Media Sharing On A Display
US20150128017A1 (en) * 2013-11-06 2015-05-07 International Business Machines Corporation Enabling interactive screenshots within collaborative applications
US20150149504A1 (en) * 2013-05-06 2015-05-28 Veeva Systems Inc. System and method for co-browsing
US9053216B1 (en) 2013-08-09 2015-06-09 Datto, Inc. CPU register assisted virtual machine screenshot capture timing apparatuses, methods and systems
US20150256594A1 (en) * 2010-08-31 2015-09-10 Mosaiqq, Inc. System and method for enabling a collaborative desktop environment
US20160210107A1 (en) * 2011-12-13 2016-07-21 Sony Computer Entertainment Inc. Information processing system, electronic device, image file playing method, and generation method
US9411789B1 (en) * 2015-08-12 2016-08-09 International Business Machines Corporation Clickable links within live collaborative web meetings
US9432740B2 (en) 2010-11-30 2016-08-30 Sony Corporation Enhanced information on mobile device for viewed program and control of internet TV device using mobile device
US20160283083A1 (en) * 2015-03-27 2016-09-29 International Business Machines Corporation Intelligent interactive screen capture
EP2751664A4 (en) * 2011-09-02 2016-11-02 Vu Lam Systems and methods for processing software application metadata associated with a software application
WO2016173422A1 (en) * 2015-04-29 2016-11-03 努比亚技术有限公司 Multi-screen interaction method and system
US9503723B2 (en) 2013-01-11 2016-11-22 Futurewei Technologies, Inc. Method and apparatus of depth prediction mode selection
US9626147B2 (en) 2013-09-04 2017-04-18 Samsung Electronics Co., Ltd. Method for controlling a display apparatus, sink apparatus thereof, mirroring system thereof
US9756091B1 (en) * 2014-03-21 2017-09-05 Google Inc. Providing selectable content items in communications
US20170289310A1 (en) * 2012-01-26 2017-10-05 Zoom International S.R.O. System and method for zero-footprint screen capture
FR3050551A1 (en) * 2016-04-26 2017-10-27 Bull Sas SCREEN COPYING METHOD
US9952749B2 (en) 2014-10-08 2018-04-24 International Business Machines Corporation Reproducing state of source environment when image was screen captured on a different computing device using resource location, resource navigation and positional metadata embedded in image
US10140382B2 (en) 2013-05-06 2018-11-27 Veeva Systems Inc. System and method for controlling electronic communications
US10437779B2 (en) * 2015-03-27 2019-10-08 International Business Machines Corporation Intelligent interactive screen capture
CN110603553A (en) * 2017-05-05 2019-12-20 微软技术许可有限责任公司 Sharing events with structured metadata
US10902081B1 (en) 2013-05-06 2021-01-26 Veeva Systems Inc. System and method for controlling electronic communications
US10963630B1 (en) * 2014-07-11 2021-03-30 Google Llc Sharing screen content in a mobile environment
US20210141933A1 (en) * 2017-11-24 2021-05-13 International Business Machines Corporation Safeguarding confidential information during a screen share session

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8635518B1 (en) 2011-07-21 2014-01-21 Google Inc. Methods and systems to copy web content selections
US9197427B2 (en) * 2011-08-26 2015-11-24 Salesforce.Com, Inc. Methods and systems for screensharing
TWI473003B (en) * 2013-03-11 2015-02-11 Awind Inc Unified multi-touch system and multi-touch method
US8924850B1 (en) * 2013-11-21 2014-12-30 Google Inc. Speeding up document loading
US8990359B1 (en) * 2014-05-19 2015-03-24 Parrable, Inc. Methods and apparatus for pixel encoded web page
GB2548223A (en) * 2016-02-08 2017-09-13 Prysm Inc Browser based snap grid
US10824594B2 (en) 2016-11-07 2020-11-03 Qualcomm Incorporated Associating a captured screenshot with application-specific metadata that defines a session state of an application contributing image data to the captured screenshot
US11379174B2 (en) * 2019-07-04 2022-07-05 Ricoh Company, Ltd. Information processing system, information processing apparatus, and information processing method
US11165726B2 (en) * 2019-10-23 2021-11-02 Sap Se Contextual sharing for screen share

Citations (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5758110A (en) * 1994-06-17 1998-05-26 Intel Corporation Apparatus and method for application sharing in a graphic user interface
US20030017928A1 (en) * 2001-07-19 2003-01-23 Bernd Meindl Roller for a running-gear mechanism or for a roller battery
US20030028503A1 (en) * 2001-04-13 2003-02-06 Giovanni Giuffrida Method and apparatus for automatically extracting metadata from electronic documents using spatial rules
US20030189601A1 (en) * 2002-04-03 2003-10-09 Microsoft Corporation Application sharing single document sharing
US20040158495A1 (en) * 2003-01-08 2004-08-12 Oracle International Corporation Methods and systems for collaborative whiteboarding and content management
US20060187228A1 (en) * 2005-01-31 2006-08-24 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Sharing including peripheral shared image device
US7200592B2 (en) * 2002-01-14 2007-04-03 International Business Machines Corporation System for synchronizing of user's affinity to knowledge
US20070168539A1 (en) * 2005-12-29 2007-07-19 Guideworks, Llc Systems and methods for managing a status change of a multimedia asset in multimedia delivery systems
US7293243B1 (en) * 2002-05-22 2007-11-06 Microsoft Corporation Application sharing viewer presentation
US7305381B1 (en) * 2001-09-14 2007-12-04 Ricoh Co., Ltd Asynchronous unconscious retrieval in a network of information appliances
US20080097985A1 (en) * 2005-10-13 2008-04-24 Fast Search And Transfer Asa Information Access With Usage-Driven Metadata Feedback
US20080097984A1 (en) * 2006-10-23 2008-04-24 Candelore Brant L OCR input to search engine
US20080189648A1 (en) * 2007-02-06 2008-08-07 Debbie Ann Anglin Attachment activation in screen captures
US20080229180A1 (en) * 2007-03-16 2008-09-18 Chicago Winter Company Llc System and method of providing a two-part graphic design and interactive document application
US7467164B2 (en) * 2002-04-16 2008-12-16 Microsoft Corporation Media content descriptions
US7580930B2 (en) * 2005-12-27 2009-08-25 Baynote, Inc. Method and apparatus for predicting destinations in a navigation context based upon observed usage patterns
US7711722B1 (en) * 2005-10-07 2010-05-04 On24, Inc. Webcast metadata extraction system and method
US20100268694A1 (en) * 2009-04-17 2010-10-21 Laurent Denoue System and method for sharing web applications
US8127365B1 (en) * 2009-11-16 2012-02-28 Trend Micro Incorporated Origination-based content protection for computer systems
US8478767B2 (en) * 2011-01-18 2013-07-02 Mark Kern Systems and methods for generating enhanced screenshots

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8046792B2 (en) 2002-03-20 2011-10-25 Tvworks, Llc Multi-channel audio enhancement for television

Patent Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5758110A (en) * 1994-06-17 1998-05-26 Intel Corporation Apparatus and method for application sharing in a graphic user interface
US20030028503A1 (en) * 2001-04-13 2003-02-06 Giovanni Giuffrida Method and apparatus for automatically extracting metadata from electronic documents using spatial rules
US20030017928A1 (en) * 2001-07-19 2003-01-23 Bernd Meindl Roller for a running-gear mechanism or for a roller battery
US7305381B1 (en) * 2001-09-14 2007-12-04 Ricoh Co., Ltd Asynchronous unconscious retrieval in a network of information appliances
US7200592B2 (en) * 2002-01-14 2007-04-03 International Business Machines Corporation System for synchronizing of user's affinity to knowledge
US20030189601A1 (en) * 2002-04-03 2003-10-09 Microsoft Corporation Application sharing single document sharing
US7467164B2 (en) * 2002-04-16 2008-12-16 Microsoft Corporation Media content descriptions
US7293243B1 (en) * 2002-05-22 2007-11-06 Microsoft Corporation Application sharing viewer presentation
US20040158495A1 (en) * 2003-01-08 2004-08-12 Oracle International Corporation Methods and systems for collaborative whiteboarding and content management
US20060187228A1 (en) * 2005-01-31 2006-08-24 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Sharing including peripheral shared image device
US7711722B1 (en) * 2005-10-07 2010-05-04 On24, Inc. Webcast metadata extraction system and method
US20080097985A1 (en) * 2005-10-13 2008-04-24 Fast Search And Transfer Asa Information Access With Usage-Driven Metadata Feedback
US7580930B2 (en) * 2005-12-27 2009-08-25 Baynote, Inc. Method and apparatus for predicting destinations in a navigation context based upon observed usage patterns
US20070168539A1 (en) * 2005-12-29 2007-07-19 Guideworks, Llc Systems and methods for managing a status change of a multimedia asset in multimedia delivery systems
US20080097984A1 (en) * 2006-10-23 2008-04-24 Candelore Brant L OCR input to search engine
US20080189648A1 (en) * 2007-02-06 2008-08-07 Debbie Ann Anglin Attachment activation in screen captures
US20080229180A1 (en) * 2007-03-16 2008-09-18 Chicago Winter Company Llc System and method of providing a two-part graphic design and interactive document application
US20100268694A1 (en) * 2009-04-17 2010-10-21 Laurent Denoue System and method for sharing web applications
US8127365B1 (en) * 2009-11-16 2012-02-28 Trend Micro Incorporated Origination-based content protection for computer systems
US8478767B2 (en) * 2011-01-18 2013-07-02 Mark Kern Systems and methods for generating enhanced screenshots
US8589423B2 (en) * 2011-01-18 2013-11-19 Red 5 Studios, Inc. Systems and methods for generating enhanced screenshots

Cited By (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150256594A1 (en) * 2010-08-31 2015-09-10 Mosaiqq, Inc. System and method for enabling a collaborative desktop environment
US9432740B2 (en) 2010-11-30 2016-08-30 Sony Corporation Enhanced information on mobile device for viewed program and control of internet TV device using mobile device
US20120317500A1 (en) * 2011-06-07 2012-12-13 At&T Intellectual Property I, L.P. System and method for data visualization and user collaboration
EP2751664A4 (en) * 2011-09-02 2016-11-02 Vu Lam Systems and methods for processing software application metadata associated with a software application
US20130111360A1 (en) * 2011-10-28 2013-05-02 Justin Kodama Accessed Location of User Interface
US9535595B2 (en) * 2011-10-28 2017-01-03 Qualcomm Incorporated Accessed location of user interface
US20160210107A1 (en) * 2011-12-13 2016-07-21 Sony Computer Entertainment Inc. Information processing system, electronic device, image file playing method, and generation method
US10255022B2 (en) 2011-12-13 2019-04-09 Sony Interactive Entertainment Inc. Information processing system, electronic device, image file playing method, and generation method for displaying a screenshot of an image file including a link to a webpage for downloading an application program
US20170289310A1 (en) * 2012-01-26 2017-10-05 Zoom International S.R.O. System and method for zero-footprint screen capture
US10484505B2 (en) * 2012-01-26 2019-11-19 ZOOM International a.s. System and method for zero-footprint screen capture
US9390403B2 (en) * 2012-02-09 2016-07-12 International Business Machines Corporation Augmented screen sharing in an electronic meeting
US9299061B2 (en) * 2012-02-09 2016-03-29 International Business Machines Corporation Augmented screen sharing in an electronic meeting
US20130212488A1 (en) * 2012-02-09 2013-08-15 International Business Machines Corporation Augmented screen sharing in an electronic meeting
US20130212490A1 (en) * 2012-02-09 2013-08-15 International Business Machines Corporation Augmented screen sharing in an electronic meeting
US20130227455A1 (en) * 2012-02-24 2013-08-29 Samsung Electronics Co. Ltd. Method of sharing content and mobile terminal thereof
US9773024B2 (en) * 2012-02-24 2017-09-26 Samsung Electronics Co., Ltd. Method of sharing content and mobile terminal thereof
US20130239020A1 (en) * 2012-03-12 2013-09-12 Samsung Electronics Co., Ltd. Electronic-book system and method for sharing additional page information thereof
US20140032698A1 (en) * 2012-07-24 2014-01-30 Cisco Technology, Inc. Utilize Extra Web Semantic for Video Caching
US9774636B2 (en) * 2012-07-24 2017-09-26 Cisco Technology, Inc. Utilize extra web semantic for video caching
WO2014084771A1 (en) * 2012-11-30 2014-06-05 Vergic Group Ab Dynamic sharing and updating of a web page
US10306266B2 (en) 2013-01-11 2019-05-28 Futurewei Technologies, Inc. Method and apparatus of depth prediction mode selection
US9503723B2 (en) 2013-01-11 2016-11-22 Futurewei Technologies, Inc. Method and apparatus of depth prediction mode selection
US9860285B2 (en) 2013-01-17 2018-01-02 Futurewei Technologies, Inc. System, apparatus, and method for sharing a screen having multiple visual components
WO2014113684A1 (en) * 2013-01-17 2014-07-24 Futurewei Technologies, Inc. System, apparatus and method for sharing a screen having multiple visual components
US9173000B2 (en) * 2013-04-12 2015-10-27 Sony Corporation Automatic discovery and mirroring of server-client remote user interface (RUI) session on a companion device and synchronously controlling both sessions using RUI on companion device
US20140310599A1 (en) * 2013-04-12 2014-10-16 Sony Corporation Automatic discovery and mirroring of server-client remote user interface (rui) session on a companion device and synchronously controlling both sessions using rui on companion device
US10831851B2 (en) * 2013-05-06 2020-11-10 Veeva Systems Inc. System and method for co-browsing
US10902081B1 (en) 2013-05-06 2021-01-26 Veeva Systems Inc. System and method for controlling electronic communications
US10140382B2 (en) 2013-05-06 2018-11-27 Veeva Systems Inc. System and method for controlling electronic communications
US10789324B2 (en) 2013-05-06 2020-09-29 Veeva Systems Inc. System and method for controlling electronic communications
US11526573B1 (en) 2013-05-06 2022-12-13 Veeva Systems Inc. System and method for controlling electronic communications
US20150149504A1 (en) * 2013-05-06 2015-05-28 Veeva Systems Inc. System and method for co-browsing
US20140331145A1 (en) * 2013-05-06 2014-11-06 Nvidia Corporation Enhancing a remote desktop with meta-information
US10169480B2 (en) 2013-05-06 2019-01-01 Veeva Systems Inc. System and method for controlling electronic communications
US10705939B2 (en) 2013-08-09 2020-07-07 Datto, Inc. Apparatuses, methods and systems for determining a virtual machine state
US9323638B2 (en) 2013-08-09 2016-04-26 Datto, Inc. Apparatuses, methods and systems for determining a virtual machine state based on CPU registers
US9053216B1 (en) 2013-08-09 2015-06-09 Datto, Inc. CPU register assisted virtual machine screenshot capture timing apparatuses, methods and systems
US9836347B2 (en) 2013-08-09 2017-12-05 Datto, Inc. Apparatuses, methods and systems for determining a virtual machine state
US20150067106A1 (en) * 2013-08-28 2015-03-05 Mersive Technologies, Inc. Graphical Indicator Of Presence, Identity, And Action For Media Sharing On A Display
US10320865B2 (en) * 2013-08-28 2019-06-11 Mersive Technologies, Inc. Graphical indicator of presence, identity, and action for media sharing on a display
US9626147B2 (en) 2013-09-04 2017-04-18 Samsung Electronics Co., Ltd. Method for controlling a display apparatus, sink apparatus thereof, mirroring system thereof
US20150128017A1 (en) * 2013-11-06 2015-05-07 International Business Machines Corporation Enabling interactive screenshots within collaborative applications
US10659499B2 (en) 2014-03-21 2020-05-19 Google Llc Providing selectable content items in communications
US9756091B1 (en) * 2014-03-21 2017-09-05 Google Inc. Providing selectable content items in communications
US11907739B1 (en) 2014-07-11 2024-02-20 Google Llc Annotating screen content in a mobile environment
US10963630B1 (en) * 2014-07-11 2021-03-30 Google Llc Sharing screen content in a mobile environment
US11347385B1 (en) 2014-07-11 2022-05-31 Google Llc Sharing screen content in a mobile environment
US11573810B1 (en) 2014-07-11 2023-02-07 Google Llc Sharing screen content in a mobile environment
US11704136B1 (en) 2014-07-11 2023-07-18 Google Llc Automatic reminders in a mobile environment
US9952749B2 (en) 2014-10-08 2018-04-24 International Business Machines Corporation Reproducing state of source environment when image was screen captured on a different computing device using resource location, resource navigation and positional metadata embedded in image
US10120542B2 (en) 2014-10-08 2018-11-06 International Business Machines Corporation Reproducing state of source environment when image was screen captured on a different computing device using resource location, resource navigation and positional metadata embedded in image
US10585566B2 (en) 2014-10-08 2020-03-10 International Business Machines Corporation Reproducing state of source environment when image was screen captured on a different computing device using resource location, resource navigation and positional metadata embedded in image
US10613717B2 (en) 2014-10-08 2020-04-07 International Business Machines Corporation Reproducing state of source environment when image was screen captured on a different computing device using resource location, resource navigation and positional metadata embedded in image
US10437779B2 (en) * 2015-03-27 2019-10-08 International Business Machines Corporation Intelligent interactive screen capture
US20160283083A1 (en) * 2015-03-27 2016-09-29 International Business Machines Corporation Intelligent interactive screen capture
WO2016173422A1 (en) * 2015-04-29 2016-11-03 努比亚技术有限公司 Multi-screen interaction method and system
US9864734B2 (en) * 2015-08-12 2018-01-09 International Business Machines Corporation Clickable links within live collaborative web meetings
US9870346B2 (en) 2015-08-12 2018-01-16 International Business Machines Corporation Clickable links within live collaborative web meetings
US20170046314A1 (en) * 2015-08-12 2017-02-16 International Business Machines Corporation Clickable links within live collaborative web meetings
US9547631B1 (en) * 2015-08-12 2017-01-17 International Business Machines Corporation Clickable links within live collaborative web meetings
US9411789B1 (en) * 2015-08-12 2016-08-09 International Business Machines Corporation Clickable links within live collaborative web meetings
FR3050551A1 (en) * 2016-04-26 2017-10-27 Bull Sas SCREEN COPYING METHOD
US10564821B2 (en) 2016-04-26 2020-02-18 Bull Sas Screen copying method
EP3239826A1 (en) * 2016-04-26 2017-11-01 Bull SAS Method for screenshot execution
CN110603553A (en) * 2017-05-05 2019-12-20 微软技术许可有限责任公司 Sharing events with structured metadata
US11783299B2 (en) 2017-05-05 2023-10-10 Microsoft Technology Licensing, Llc Sharing events with structured metadata
US11455423B2 (en) * 2017-11-24 2022-09-27 International Business Machines Corporation Safeguarding confidential information during a screen share session
US20210141933A1 (en) * 2017-11-24 2021-05-13 International Business Machines Corporation Safeguarding confidential information during a screen share session

Also Published As

Publication number Publication date
WO2011101246A1 (en) 2011-08-25
US20120203854A1 (en) 2012-08-09
US9021367B2 (en) 2015-04-28
TW201203097A (en) 2012-01-16

Similar Documents

Publication Publication Date Title
US9021367B2 (en) Metadata capture for screen sharing
US11799984B2 (en) Installable web applications
US10601633B2 (en) Virtual window screen renderings using application connectors
KR101507629B1 (en) Browser-based proxy server for customization and distribution of existing applications
US8924943B2 (en) Browser emulator system
US9542501B2 (en) System and method for presenting content in a client/server environment
US10671698B2 (en) Language translation using embeddable component
US8732588B2 (en) Method and apparatus for remotely displaying screen files and efficiently handling remote operator input
US9405745B2 (en) Language translation using embeddable component
US9436482B2 (en) Input content to application via web browser
US20080228504A1 (en) Technique to Deflect Incident Ticket Submission in Real-Time
US7500181B2 (en) Method for updating a portal page
US8880993B2 (en) Handling unexpected responses to script executing in client-side application
US20120259964A1 (en) Cloud computing method capable of hiding real file paths
US20110202850A1 (en) Automatic Removal of Sensitive Information from a Computer Screen
US20060230059A1 (en) Method and apparatus to select and deliver portable portlets
GB2494027A (en) Enabling a web application to call at least one native function of a mobile device
US20100229081A1 (en) Method for Providing a Navigation Element in an Application
US20160359989A1 (en) Recording And Triggering Web And Native Mobile Application Events With Mapped Data Fields
JP2013528859A (en) Integration of client application and web page
US10142446B2 (en) Dialog server
US20190230188A1 (en) Hosted application as web widget toolkit
Koehl et al. M. site: Efficient content adaptation for mobile devices
CN112596838B (en) Method, device, equipment and storage medium for displaying universal Web page
CN113630434A (en) Processing method for client access request and server

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHAN, YUK;TO, TIN H.;TSANG, ANDREW;AND OTHERS;REEL/FRAME:023947/0039

Effective date: 20100215

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE