US20040004619A1 - Method and apparatus for extending structured content to support streaming - Google Patents
Method and apparatus for extending structured content to support streaming Download PDFInfo
- Publication number
- US20040004619A1 US20040004619A1 US10/175,641 US17564102A US2004004619A1 US 20040004619 A1 US20040004619 A1 US 20040004619A1 US 17564102 A US17564102 A US 17564102A US 2004004619 A1 US2004004619 A1 US 2004004619A1
- Authority
- US
- United States
- Prior art keywords
- scene
- time
- data
- stream
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1101—Session protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234318—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into objects, e.g. MPEG-4 objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/30—Definitions, standards or architectural aspects of layered protocol stacks
- H04L69/32—Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
- H04L69/322—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
- H04L69/329—Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99933—Query processing, i.e. searching
Definitions
- the teachings of this invention relate generally to data transmission protocols and formats and, more specifically, relate to extensions for arbitrarily structured data to enable streaming in low-bandwidth and other environments.
- Structured data includes content such as databases, spreadsheets, address books, configuration parameters, financial transactions and technical drawings.
- the Extensible Markup Language, or XML is a standard text-based representation of structured data on the World Wide Web, and has become the most popular format to represent structured data.
- An example XML content is as follows: ⁇ person> ⁇ name>John Smith ⁇ /name> ⁇ street>24 Main Street ⁇ /street> ⁇ city>Old Town ⁇ /city> ⁇ state>TX ⁇ /state> ⁇ zip>75039 ⁇ /zip> ⁇ /person>
- Vector graphics describes two-dimensional graphics, and allows for three types of graphic objects: vector graphic shapes (e.g., paths consisting of straight lines and curves), images and text. Graphical objects can be grouped, styled, transformed and composited into previously rendered objects. Vector graphics drawings can be dynamic and interactive.
- W3C Scalable Vector Graphics
- SVG Scalable Vector Graphics
- W3C Consortium Scalable Vector Graphics Specification
- WWW Candidate Recommendation 2000.
- XML was designed to be useful in the World Wide Web environment and assumes that sufficient bandwidth exists.
- relatively low-bandwidth communications e.g., mobile communication applications
- Streaming, or substantially continuous data transmission, is crucial for most applications that require the delivery of large content, e.g., a large cartoon animation represented using XML.
- the streaming of non-structured video and audio data over various data networks is the intent of a set of standards and products (e.g., MPEG-4 and RealNetworksTM).
- audio/video players are usually designed to begin playing received content from the Internet without requiring one to first download the entire file.
- Macromedia FlashTM presents an approach for the streaming of vector graphics content as binary data. FlashTM defines a “display list” method consisting of a set of commands for each frame, and may thus be viewed as an alternative to the structured representation of graphics content.
- U.S. Pat. No. 6,057,854 “System and method of providing interactive vector graphics over a network”, John Davis, Jr. And Scott M. Glazer, discloses a client-server based system where an interactive vector object that is downloadable over a network includes a vector object and an active area defined by the vector object.
- a vector graphics network file in connection with the data, renders the image of the vector object on a client system that is connectable to the network.
- MPEG-7 BiM Bitmap Format for MPEG-7 ISO/IEC 15938-1
- MPEG-7 BiM Bitmap Format for MPEG-7 ISO/IEC 15938-1
- BiM defines a compression method, and does not consider how to extend the structure content to allow streaming.
- a method for streaming XML data is presented in WO 01/26373 A2, “Method and Apparatus for Streaming XML Content”, Raymond J. Krasinski.
- Each XML document is decomposed and encoded as a collection of sub-trees.
- Each sub-tree from the larger XML document tree can be parsed and validated by the XML receiver as if it is and independent tree.
- the structure node indicates the relationship of the sub-tree to other sub-trees, thereby allowing the XML receiver to reconstruct the full tree.
- a method for transferring data that is somewhat analogous to streaming is the HTML Push/Pull technology.
- a Server Push operation the server sends down a packet of data and the browser displays the data but leaves the connection open. Whenever the server desires it sends more data and the browser displays the data, still leaving the connection open.
- a Client Pull operation the server sends down a block of data, including a directive (in the HTTP response or the document header) that specifies “reload this data in x seconds”, or “go load another URL in y seconds”. After the specified amount of time has elapsed, the client performs the directed operation, either reloading the current data or retrieving new data.
- the HTTP connection is held open for an indefinite period of time (until the server knows it is done sending data to the client and sends a terminator, or until the client interrupts the connection).
- the HTTP connections are never held open; rather, the client is instructed when to open a new connection, and what data to fetch when it does so.
- SMIL Synchronized Multimedia Integration Language
- This invention provides a method for the extension of arbitrary structured data to support streaming.
- a presently preferred embodiment of the method, and of apparatus that implements the method splits or partitions the content into a set of scenes that are capable of being represented and processed independently of each other (e.g., scenes in a cartoon animation).
- Each scene contains a list of objects (e.g., cartoon characters, background).
- the objects are considered to be atomic (non-partitionable) units that combine content and that can be processed collectively.
- the method further defines addition, deletion and modification commands that operate on the objects during streaming.
- the invention provides a means to convert any arbitrarily structured data to a streamable form by extending and reorganizing the structure of the data.
- a first advantage is that the decoder in the data receiver is not required to have the complete data structure to process it, as each scene element is self-contained for representation.
- a second advantage is that arbitrary objects can be added or deleted to the data structure at any time.
- a further advantage is that the method of this invention supports multiple name spaces, as it is a namespace independent extension of the document.
- a method for sending structured data from a transmitter to a receiver through a network environment comprises partitioning the structured data into at least one scene by inserting scene delimiter statements, where each scene includes at least one object; within a scene, opening a stream by inserting a first stream delimiter command; within the stream, using at least one of an object addition, deletion and modification command; within the scene, closing the open stream by inserting a second stream delimiter command; and transmitting the resulting structured data through the network environment.
- a scene includes an attribute that specifies a total duration of the scene, and overrides a total scene duration determined from elements that comprise the scene.
- a plurality of scenes cover mutually exclusive time intervals, and a union of the time intervals forms a continuous time interval.
- the object addition command references an object in a current graphics tree, and includes attributes of anchor, relation and time.
- the anchor attribute is a reference to an ID of an object in the current graphics tree and, with the relation attribute, specifies where in the current graphics tree an associated object should be inserted, and the time attribute specifies the time when the object insertion is to be performed.
- the object deletion command references an object in a current graphics tree, and includes attributes of anchor and time.
- the anchor attribute is a reference to an ID of an object in the current graphics tree that is to be removed from the current graphics tree, and the time attribute specifies the time when the object removal is to be performed.
- the object modification command includes a refresh time interval when the associated object is to be updated with a new object downloaded from a link refresh network address.
- the structured data may be Extensible Markup Language (XML) data, which in turn may comprise Scalable Graphics Format SVG data.
- XML Extensible Markup Language
- SVG Scalable Graphics Format
- the network environment includes a wireless link, such as an RF link between a network operator and a mobile station.
- the data receiver can comprise the mobile station with a display for displaying a result of processing of the partitioned structured data received by the mobile station receiver.
- FIG. 1 is a simplified diagram of a technical environment of this invention
- FIG. 2A illustrates a reorganization of structured data for streaming, where FIG. 2A shows a directed graph representation of structured data and FIG. 2B shows the effect of adding the steaming extensions in accordance with this invention
- FIGS. 3 A- 3 G illustrate a usage scenario that is one non-limiting example of the operation of the method and apparatus in accordance with this invention
- FIG. 4 is a simplified block diagram of an embodiment of this invention in a wireless network containing a network operator, a mobile station and a wireless (RF) link network environment;
- FIG. 5A illustrates an example of a streamed time-based animation
- FIG. 5B illustrates an example of a streamed frame-based animation
- the method in accordance with this invention can be summarized as providing enhancements to an existing structured content to support streamed content delivery and representation from a data transmitter 1 through a network environment 2 to a data receiver 3 .
- a first step defines an XML namespace, which supports streamable components as well as formatted objects. This name space is utilized to organize the objects in a structured manner.
- the distinct elements and attributes have the following scope hierarchy and, for the sake of illustration, define a namespace sml (Stream Modeling Language) as follows: ⁇ sml:sm/> ⁇ sml:scene> ⁇ sml:stream> ⁇ sml:deleteObject> ⁇ sml:addObject>
- a sml document in accordance with this invention includes one or more scenes, with each scene covering mutually exclusive time intervals.
- the union of the time intervals (scenes) comprises a continuous time interval.
- each scene contains one self-contained object.
- Each such object is defined to be associated with zero or one stream object.
- a ‘scene’ has one attribute: ‘dur’. This attribute specifies the total duration of the scene, and overrides (clips) the total duration determined by the elements within the scene. If the ‘dur’ attribute is not present the scene is static and continuously present.
- a scene object can contain one or more stream objects.
- a stream object contains at least one object that performs an add, a delete or a refresh operation.
- Each stream content is transmitted to the receiver 3 , after the termination of an initial transmission of the static content of the sml document, in-line with time values defined in addObject, deleteObject elements and refresh attributes (defined below).
- a stream object has no attributes.
- the deleteObject command is referenced to an existing graphics object.
- the deleteObject command has two required attributes: target and time.
- the target attribute is a reference to the identification (ID) of an object to be deleted
- the time attribute is the time when the object deletion operation is to be performed.
- the time attributes of consecutive deleteObject (and addObject) commands are defined to make a non-decreasing sequence.
- the addObject command may reference any object within the graphics format scope, with a reference to an existing graphics tree.
- the addObject command has three required attributes: anchor, relation and time.
- the anchor attribute is a reference to the ID of an object in the current graphics tree. Together with the relation attribute, the anchor attribute specifies where in the graphics tree the object(s) to be declared should be inserted.
- the possible values of relation are (a) child, (b) prevSibling and nextSibling.
- the time attribute is the time when the object insertion is to be performed.
- a refreshRate is the time interval when the object should be updated with a new object downloaded from a link refreshURL, and can be added to all xlink:href elements in any namespace.
- This invention can be implemented by extending existing structured data.
- a non-limiting example is shown below of how the Scalable Graphics Format (SVG) is extended for including the case of a streaming animation.
- SVG Scalable Graphics Format
- the first ⁇ scene> statement opens a scene, and the following ⁇ svg> code is downloaded to the data receiver 3 in order to setup and begin the animation.
- the statements and commands between the two ⁇ stream> statements mark a stream, where the first ⁇ stream> statement opens the stream and the second ⁇ stream> statement closes the stream.
- addObject and deleteObject commands in accordance with this invention, where one of the addObject commands adds an object obtained and refreshed from a network address, in this example the URL www.example.foo/stockupdate.svg.
- the second ⁇ scene> statement closes the first scene.
- the second scene where an ellipse is added to the displayed animation, does not include a stream.
- FIGS. 2A and 2B show an example of the reorganization of structured data for streaming, and should be viewed in conjunction with the following pseudo-code representations.
- FIGS. 3A, 3B, 3 C, 3 D, 3 E, 3 F and 3 G are an example of a usage scenario for the immediately preceding streaming pseudo-code embodiment, where FIG. 3A shows the content on the data transmitter (server) 1 at all times; FIG. 3B shows the content at the data receiver (client) 3 at time 0; FIG. 3C shows the content at the data receiver 3 at time 1 second. where the added element 315 is updated once per second; FIG. 3D shows the content at the data receiver 3 at time 5 seconds, with added elements 306 , 307 and 308 ; FIG. 3E shows the content at the data receiver 3 at time 15 seconds, with deleted elements 302 , 303 , 304 and 315 ; and FIG.
- FIG. 3F shows the content at the data receiver 3 at time 25 seconds, with deleted elements 316 , 317 and 309 .
- FIG. 3G illustrates a series of “snapshots” taken at time values 0, 1 second, 3 seconds, 5 seconds, 10 seconds, 15 seconds and 25 seconds, and depicts the animation that the client or data receiver 3 renders on a display device in response to the receipt of the foregoing streaming pseudo-code through the network environment 2 .
- FIG. 4 illustrates a simplified block diagram of an embodiment of a wireless communications system 5 that is suitable for practicing this invention.
- the wireless communications system 5 includes at least one mobile station (MS) 100 .
- FIG. 4 also shows an exemplary network operator having, for example, a GPRS Support Node (GSN) 30 for connecting to a telecommunications network, such as a Public Packet Data Network or PDN, at least one base station controller (BSC) 40 , and a plurality of base transceiver stations (BTS) 50 that transmit in a forward or downlink direction both physical and logical channels over a wireless link to the mobile station 100 in accordance with a predetermined air interface standard.
- GSN GPRS Support Node
- BSC base station controller
- BTS base transceiver stations
- a reverse or uplink communication path also exists from the mobile station 100 to the network operator, which conveys mobile originated access requests and traffic.
- the wireless link represents, for the purposes of this description, a limited bandwidth network connection, and corresponds in whole or in part to the network environment 2 shown in FIG. 4.
- the network environment 2 could be implemented through, as examples, a wire, or a cable, or an optical fiber, or through an optical wireless link (e.g., an infra-red link).
- the air interface standard can conform to any suitable standard or protocol, and may enable both voice and data traffic, such as data traffic enabling Internet 70 access and downloads.
- the air interface standard can be a Time Division Multiple Access (TDMA) air interface that supports a GSM or an advanced GSM protocol and air interface, or it may be a Code Division Multiple Access (CDMA) air interface standard.
- TDMA Time Division Multiple Access
- CDMA Code Division Multiple Access
- the network operator includes or is coupled to a Streaming Content Server (SCS) 60 that transmits streaming content towards the mobile station 100 , via the BSC 40 and BTS 50 , which together may be considered to be equivalent to the data transmitter 1 shown in FIG. 1, while the mobile station 100 may be considered to be equivalent to the data receiver or client 3 of FIG. 1.
- SCS 60 operates in accordance with the foregoing description of the presently preferred embodiments of this invention to stream structured data towards the mobile station 100 , and the mobile station 100 , as the client, is responsive to the received stream to decode same and render the graphical content, as was illustrated above in reference to the example shown in FIGS. 3 B- 3 G.
- the mobile station 100 typically includes a microcontrol unit (MCU) 120 having an output coupled to an input of a display 140 and an input coupled to an output of a keyboard or keypad 160 .
- the mobile station 100 may be a handheld radiotelephone, such as a cellular telephone or a personal communicator.
- the mobile station 100 could also be contained within a card or module that is connected during use to another device.
- the mobile station 10 could be contained within a PCMCIA or similar type of card or module that is installed during use within a portable data processor, such as a laptop or notebook computer, or even a computer that is wearable by the user.
- the MCU 120 is assumed to include or be coupled to some type of a memory 130 , including a read-only memory (ROM) for storing an operating program, as well as a random access memory (RAM) for temporarily storing required data, scratchpad memory, received packet data, packet data to be transmitted, and the like.
- ROM read-only memory
- RAM random access memory
- SIM separate, removable SIM (not shown) can be provided as well, the SIM storing, for example, a preferred Public Land Mobile Network (PLMN) list and other subscriber-related information.
- PLMN Public Land Mobile Network
- the ROM is assumed, for the purposes of this invention, to store a program enabling the MCU 120 to execute the software routines, layers and protocols required to implement the data receiver 3 methods in accordance with these teachings, and may as well provide a suitable user interface (UI), via a display 140 and keypad 160 , with a user.
- UI user interface
- the result of the rendering of the decoded structured data stream can be displayed on the display 140 , as was noted above in the discussion of FIG. 3G.
- a microphone and speaker are typically provided for enabling the user to conduct voice calls in a conventional manner.
- the mobile station 100 also contains a wireless section that includes a digital signal processor (DSP) 180 , or equivalent high speed processor or logic, as well as a wireless transceiver that includes a transmitter 200 and a receiver 220 , both of which are coupled to an antenna 240 for communication with the network operator.
- DSP digital signal processor
- LO local oscillator
- the mobile station 100 also contains a wireless section that includes a digital signal processor (DSP) 180 , or equivalent high speed processor or logic, as well as a wireless transceiver that includes a transmitter 200 and a receiver 220 , both of which are coupled to an antenna 240 for communication with the network operator.
- At least one local oscillator (LO) 260 such as a frequency synthesizer, is provided for tuning the transceiver.
- Data such as digitized voice data
- the structured data stream from the Streaming Content Server (SCS) 60 is also received through the antenna 240 , and is thereafter processed by the DSP 180 and/or the MCU 120 to display the encoded animation or other video images to the user on the display 160 .
- SCS Streaming Content Server
- this invention overcomes the problems that are inherent in the prior art approaches discussed previously. For example, and with regard to the streaming of non-structured video and audio data, these formats are suited for streaming only non-structured binary data, such as video and audio, and not structured data.
- this approach uses a binary file format and cannot be used for generic structured data or XML.
- this invention enables the streaming of arbitrarily structured data.
- This invention can also be used to stream vector graphics using time-based animation.
- the example shown in FIG. 5 illustrates significant differences between streaming using time based animation versus the frame based approach.
- the boxes shown in FIG. 5A illustrate a streamed, time based animation content.
- the boxes of FIG. 5B illustrate a streamed, frame based animation.
- the frame-based approach requires streaming new coordinate values at each frame.
- time-based animation requires streaming only keyframes of the animation.
- the use of the teachings of this invention enables one to stream time-based animations, as well as any arbitrary time-based type of content.
- the MPEG-7 BiM approach defines a compression and streaming method for structured data; but it does not define how the input data is partitioned into streamable subsets. This invention both addresses and solves this problem.
- the Push and Pull approach is not in and of itself a method to organize structured data, but instead represent a functionality of the HTML language and its transfer protocol HTTP.
- This invention solves a different problem than HTTP Push and Pull, as it enables the content creator to structure the data to be streamed.
- the SMIL technique has two elements ‘par’ (parallel) and ‘seq’ (sequential) that handle the timing and execution of SMIL content.
- these two elements do not introduce or define a streaming functionality.
- SMIL enables the simple authoring of interactive audiovisual presentations, and is typically used for rich media/multimedia presentations that integrate streaming audio and video with images, text or any other media type. In all these cases the actual streaming is file format dependent and not related to the functionality of SMIL per se.
- the teachings of this invention can be applied to a workstation of a graphical content developer, such as one that forms a part of, or that is coupled to, the SCS 60 of FIG. 4, or that is associated with the receiver 3 .
- the content developer may divide a pre-existing or a new temporally-based structured data graphical presentation, such as an animation, into smaller temporally-based scenes each having some temporal extension. These scenes are not, per se, equivalent to the conventional use of frames. In fact, some number of conventional frames can be incorporated within a single scene, thereby reducing the amount of structured data content that must be transmitted, processed and stored.
- teachings enable a large and complex graphical presentation, such as an animation, to be efficiently transmitted to and displayed by a relatively small device, such as a cellular telephone, a personal communicator or a personal digital assistant (PDA).
- a relatively small device such as a cellular telephone, a personal communicator or a personal digital assistant (PDA).
- PDA personal digital assistant
Abstract
Description
- The teachings of this invention relate generally to data transmission protocols and formats and, more specifically, relate to extensions for arbitrarily structured data to enable streaming in low-bandwidth and other environments.
- Information is typically represented using a structured description. Structured data includes content such as databases, spreadsheets, address books, configuration parameters, financial transactions and technical drawings. The Extensible Markup Language, or XML, is a standard text-based representation of structured data on the World Wide Web, and has become the most popular format to represent structured data. An example XML content is as follows:
<person> <name>John Smith</name> <street>24 Main Street</street> <city>Old Town</city> <state>TX</state> <zip>75039</zip> </person> - One particular application of XML is for representing vector graphics data. Vector graphics describes two-dimensional graphics, and allows for three types of graphic objects: vector graphic shapes (e.g., paths consisting of straight lines and curves), images and text. Graphical objects can be grouped, styled, transformed and composited into previously rendered objects. Vector graphics drawings can be dynamic and interactive.
- One organization (W3C) has defined a standard vector graphics language in XML, referred to as the Scalable Vector Graphics or SVG, W3C Consortium, Scalable Vector Graphics Specification, WWW Candidate Recommendation, 2000.
- An example of structured SVG content for representing vector graphics data is as follows:
<svg width=“5cm” height=“5cm” xmlns=“http://www.w3.org/2000/svg”> <desc>A group with two rectangles </desc> <g id=“group1” fill=“red”> <rect x=“1cm” y=“1cm” width=“1cm” height=“1cm”/> <rect x=“3cm” y=“1cm” width=“1cm” height=“1cm”/> </g> </svg> - XML was designed to be useful in the World Wide Web environment and assumes that sufficient bandwidth exists. However, relatively low-bandwidth communications (e.g., mobile communication applications) require a form of incremental representation, to decrease latency of a user agent's processing of the data, and to decrease memory requirements. Streaming, or substantially continuous data transmission, is crucial for most applications that require the delivery of large content, e.g., a large cartoon animation represented using XML.
- Prior to this invention problems have existed in converting formatted data to a form that can be streamed with minimal change to the original content. In addition, problems existed in excessive bandwidth and storage requirements for transmitting and processing large structured content.
- Prior attempts to address these problems include the following.
- The streaming of non-structured video and audio data over various data networks is the intent of a set of standards and products (e.g., MPEG-4 and RealNetworks™). For example, audio/video players are usually designed to begin playing received content from the Internet without requiring one to first download the entire file.
- Another approach, Macromedia Flash™ (www.macromedia.com), presents an approach for the streaming of vector graphics content as binary data. Flash™ defines a “display list” method consisting of a set of commands for each frame, and may thus be viewed as an alternative to the structured representation of graphics content.
- U.S. Pat. No. 6,057,854, “System and method of providing interactive vector graphics over a network”, John Davis, Jr. And Scott M. Glazer, discloses a client-server based system where an interactive vector object that is downloadable over a network includes a vector object and an active area defined by the vector object. A vector graphics network file, in connection with the data, renders the image of the vector object on a client system that is connectable to the network.
- Another approach, MPEG-7 BiM (Binary format for MPEG-7 ISO/IEC 15938-1), is a binary format for the compression and streaming of XML Documents. MPEG-7 BiM is based on dividing a large XML document into pieces and delivering them separately to the client. BiM defines a compression method, and does not consider how to extend the structure content to allow streaming.
- A method for streaming XML data is presented in WO 01/26373 A2, “Method and Apparatus for Streaming XML Content”, Raymond J. Krasinski. Each XML document is decomposed and encoded as a collection of sub-trees. Each sub-tree from the larger XML document tree can be parsed and validated by the XML receiver as if it is and independent tree. The structure node indicates the relationship of the sub-tree to other sub-trees, thereby allowing the XML receiver to reconstruct the full tree.
- A method for transferring data that is somewhat analogous to streaming is the HTML Push/Pull technology. In a Server Push operation the server sends down a packet of data and the browser displays the data but leaves the connection open. Whenever the server desires it sends more data and the browser displays the data, still leaving the connection open. In a Client Pull operation the server sends down a block of data, including a directive (in the HTTP response or the document header) that specifies “reload this data in x seconds”, or “go load another URL in y seconds”. After the specified amount of time has elapsed, the client performs the directed operation, either reloading the current data or retrieving new data.
- In the Server Push operation the HTTP connection is held open for an indefinite period of time (until the server knows it is done sending data to the client and sends a terminator, or until the client interrupts the connection). In the Client Pull operation the HTTP connections are never held open; rather, the client is instructed when to open a new connection, and what data to fetch when it does so.
- The Synchronized Multimedia Integration Language (SMIL) (http://www.w3.org/TR/smil20/) enables the simple authoring of interactive audiovisual presentations. SMIL is typically used for multimedia presentations that integrate streaming audio and video with images, text or any other media type.
- None of these prior art approaches has been found to adequately solve the problem providing structured data in a form that allows streaming in low bandwidth and other environments.
- The foregoing and other problems are overcome, and other advantages are realized, in accordance with the presently preferred embodiments of these teachings.
- This invention provides a method for the extension of arbitrary structured data to support streaming. A presently preferred embodiment of the method, and of apparatus that implements the method, splits or partitions the content into a set of scenes that are capable of being represented and processed independently of each other (e.g., scenes in a cartoon animation). Each scene contains a list of objects (e.g., cartoon characters, background). The objects are considered to be atomic (non-partitionable) units that combine content and that can be processed collectively. The method further defines addition, deletion and modification commands that operate on the objects during streaming.
- The invention provides a means to convert any arbitrarily structured data to a streamable form by extending and reorganizing the structure of the data. There are a number of advantages that are realized by the practice of the teachings of this invention, as compared to the other known alternatives.
- A first advantage is that the decoder in the data receiver is not required to have the complete data structure to process it, as each scene element is self-contained for representation.
- A second advantage is that arbitrary objects can be added or deleted to the data structure at any time.
- A further advantage is that the method of this invention supports multiple name spaces, as it is a namespace independent extension of the document.
- A method is disclosed for sending structured data from a transmitter to a receiver through a network environment. The method comprises partitioning the structured data into at least one scene by inserting scene delimiter statements, where each scene includes at least one object; within a scene, opening a stream by inserting a first stream delimiter command; within the stream, using at least one of an object addition, deletion and modification command; within the scene, closing the open stream by inserting a second stream delimiter command; and transmitting the resulting structured data through the network environment.
- A scene includes an attribute that specifies a total duration of the scene, and overrides a total scene duration determined from elements that comprise the scene. A plurality of scenes cover mutually exclusive time intervals, and a union of the time intervals forms a continuous time interval.
- The object addition command references an object in a current graphics tree, and includes attributes of anchor, relation and time. The anchor attribute is a reference to an ID of an object in the current graphics tree and, with the relation attribute, specifies where in the current graphics tree an associated object should be inserted, and the time attribute specifies the time when the object insertion is to be performed.
- The object deletion command references an object in a current graphics tree, and includes attributes of anchor and time. The anchor attribute is a reference to an ID of an object in the current graphics tree that is to be removed from the current graphics tree, and the time attribute specifies the time when the object removal is to be performed.
- The object modification command includes a refresh time interval when the associated object is to be updated with a new object downloaded from a link refresh network address.
- The structured data may be Extensible Markup Language (XML) data, which in turn may comprise Scalable Graphics Format SVG data.
- In the presently preferred embodiment, the network environment includes a wireless link, such as an RF link between a network operator and a mobile station. In this case the data receiver can comprise the mobile station with a display for displaying a result of processing of the partitioned structured data received by the mobile station receiver.
- The foregoing and other aspects of these teachings are made more evident in the following Detailed Description of the Preferred Embodiments, when read in conjunction with the attached Drawing Figures, wherein:
- FIG. 1 is a simplified diagram of a technical environment of this invention;
- FIGS. 2A and 2B, collectively referred to as FIG. 2, illustrate a reorganization of structured data for streaming, where FIG. 2A shows a directed graph representation of structured data and FIG. 2B shows the effect of adding the steaming extensions in accordance with this invention;
- FIGS.3A-3G illustrate a usage scenario that is one non-limiting example of the operation of the method and apparatus in accordance with this invention;
- FIG. 4 is a simplified block diagram of an embodiment of this invention in a wireless network containing a network operator, a mobile station and a wireless (RF) link network environment; and
- FIG. 5A illustrates an example of a streamed time-based animation, while FIG. 5B illustrates an example of a streamed frame-based animation.
- By way of introduction, and referring to FIG. 1, the method in accordance with this invention can be summarized as providing enhancements to an existing structured content to support streamed content delivery and representation from a data transmitter1 through a
network environment 2 to adata receiver 3. - The approach is as follows. A first step defines an XML namespace, which supports streamable components as well as formatted objects. This name space is utilized to organize the objects in a structured manner. The distinct elements and attributes have the following scope hierarchy and, for the sake of illustration, define a namespace sml (Stream Modeling Language) as follows:
<sml:sm/> <sml:scene> <sml:stream> <sml:deleteObject> <sml:addObject> - Turning first to the definition and attributes of a scene, a sml document in accordance with this invention includes one or more scenes, with each scene covering mutually exclusive time intervals. The union of the time intervals (scenes) comprises a continuous time interval.
- Preferably, each scene contains one self-contained object. Each such object is defined to be associated with zero or one stream object. A ‘scene’ has one attribute: ‘dur’. This attribute specifies the total duration of the scene, and overrides (clips) the total duration determined by the elements within the scene. If the ‘dur’ attribute is not present the scene is static and continuously present.
- Turning now to the definition and attributes of ‘stream’, a scene object can contain one or more stream objects. A stream object contains at least one object that performs an add, a delete or a refresh operation. Each stream content is transmitted to the
receiver 3, after the termination of an initial transmission of the static content of the sml document, in-line with time values defined in addObject, deleteObject elements and refresh attributes (defined below). A stream object has no attributes. - With regard to the definition and attributes of a deleteObject command, the deleteObject command is referenced to an existing graphics object. The deleteObject command has two required attributes: target and time. The target attribute is a reference to the identification (ID) of an object to be deleted, and the time attribute is the time when the object deletion operation is to be performed. The time attributes of consecutive deleteObject (and addObject) commands are defined to make a non-decreasing sequence.
- With regard to the definition and attributes of an addObject command, the addObject command may reference any object within the graphics format scope, with a reference to an existing graphics tree. The addObject command has three required attributes: anchor, relation and time.
- The anchor attribute is a reference to the ID of an object in the current graphics tree. Together with the relation attribute, the anchor attribute specifies where in the graphics tree the object(s) to be declared should be inserted. The possible values of relation are (a) child, (b) prevSibling and nextSibling. The time attribute is the time when the object insertion is to be performed.
- For example, if there is a group object in the graphics tree with an ID of “group1”, the following command adds a rectangle as the last child of this group at time=5 seconds:
<addObject time=“5s” anchor=“group 1” relation=“child”> <rectangle x=“0” y=“5” width=“5” height=“10”/> </addObject> - As was stated above, the time attributes of consecutive addObject and deleteObject commands should form a non-decreasing sequence.
- With regard now to Refresh Attributes, a refreshRate is the time interval when the object should be updated with a new object downloaded from a link refreshURL, and can be added to all xlink:href elements in any namespace.
- This invention can be implemented by extending existing structured data. For illustration purposes, a non-limiting example is shown below of how the Scalable Graphics Format (SVG) is extended for including the case of a streaming animation.
<sml> <scene> <svg id=“main” height=“100” width=“100”> <g id=“g1”> <path fill=“red” d= “M10 10 L 100 10 100 50 10 50z”/> <animateTransform attributeName=“transform” type=“scale” dur=“0.8s” from= “.7” to=“1.1” repeatCount=“indefinite”/> </g> </svg> *********** <stream> <addObject time=“5s” anchor=“g1” relation= “nextSibling”> <g id=“g2”> <rect fill=“blue” x=“10” y=“15” width=“10” height=“10”/> <animateTransform attributeName=“transform” type=“translate” dur=“15s” from=“0” to=“80” repeatCount= “indefinite”/> </g> </addObject> <deleteObject time=“15s” target=“g1”> <addObject time=“1s” anchor=“g1” relation=“child” refreshRate=“1s” refreshURL=www.example.foo/stockupdate.svg>> </addObject> *********** </stream> </scene> <scene dur=“25”> <svg> <ellipse cx=“30” cy=“30” rx=“10” ry=“20”/> </svg> </scene> </sml> - The first <scene> statement opens a scene, and the following <svg> code is downloaded to the
data receiver 3 in order to setup and begin the animation. In accordance with an aspect of this invention the statements and commands between the two <stream> statements (marked with asterisks) mark a stream, where the first <stream> statement opens the stream and the second <stream> statement closes the stream. In between are addObject and deleteObject commands in accordance with this invention, where one of the addObject commands adds an object obtained and refreshed from a network address, in this example the URL www.example.foo/stockupdate.svg. The second <scene> statement closes the first scene. The second scene, where an ellipse is added to the displayed animation, does not include a stream. - Note in this example how the <scene> statements are used to partition the structured SVG data into independent scenes, where each scene contains a list of objects used to combine content and that can be processed collectively. Note as well the object addition, deletion and modification commands that operate on objects during streaming.
- FIGS. 2A and 2B show an example of the reorganization of structured data for streaming, and should be viewed in conjunction with the following pseudo-code representations.
- The following is a pseudo-code representation of conventional Structured Data with SVG Element Names (FIG. 2A):
<svg> (301) <g> (302) <path/> (303) <animateTransform/> (304) <image xlink:href= =www.example.foo/stockupdate.svg/> (305) </g> <g> (306) <rect/> (307) <animateTransform/> (308) </g> <ellipse/> (309) </svg> - The following is a pseudo-code representation of Structured Data with Streaming Extensions in accordance with this invention (FIG. 2B):
<sml>(310) <scene>(311) <svg>(301) <g id=“g1”>(302) <path/>(303) <animateTransform/>(304) </g> </svg> <stream>(312) <addObject time=“5s” anchor=“g1” relation= “nextSibling”>(313) <g id=“g2”>(306) <rect/>(307) <animateTransform/>(308) </g> </addObject> <deleteObject time=“15s” target=“g1”>(314) <addObject time=“1s” anchor=“g1” relation=“child” refreshRate=“1s” refreshURL=www.example.foo/stockupdate.svg>(315) </addObject> </stream> </scene> <scene dur=“25”>(316) <svg>(317) <ellipse/>(309) </svg> </scene> </sml> - FIGS. 3A, 3B,3C, 3D, 3E, 3F and 3G are an example of a usage scenario for the immediately preceding streaming pseudo-code embodiment, where FIG. 3A shows the content on the data transmitter (server) 1 at all times; FIG. 3B shows the content at the data receiver (client) 3 at time 0; FIG. 3C shows the content at the
data receiver 3 at time 1 second. where the addedelement 315 is updated once per second; FIG. 3D shows the content at thedata receiver 3 attime 5 seconds, with addedelements data receiver 3 at time 15 seconds, with deletedelements data receiver 3 at time 25 seconds, with deletedelements data receiver 3 renders on a display device in response to the receipt of the foregoing streaming pseudo-code through thenetwork environment 2. - FIG. 4 illustrates a simplified block diagram of an embodiment of a
wireless communications system 5 that is suitable for practicing this invention. Thewireless communications system 5 includes at least one mobile station (MS) 100. FIG. 4 also shows an exemplary network operator having, for example, a GPRS Support Node (GSN) 30 for connecting to a telecommunications network, such as a Public Packet Data Network or PDN, at least one base station controller (BSC) 40, and a plurality of base transceiver stations (BTS) 50 that transmit in a forward or downlink direction both physical and logical channels over a wireless link to themobile station 100 in accordance with a predetermined air interface standard. A reverse or uplink communication path also exists from themobile station 100 to the network operator, which conveys mobile originated access requests and traffic. The wireless link represents, for the purposes of this description, a limited bandwidth network connection, and corresponds in whole or in part to thenetwork environment 2 shown in FIG. 4. In other embodiments of this invention thenetwork environment 2 could be implemented through, as examples, a wire, or a cable, or an optical fiber, or through an optical wireless link (e.g., an infra-red link). - The air interface standard can conform to any suitable standard or protocol, and may enable both voice and data traffic, such as data
traffic enabling Internet 70 access and downloads. As non-limiting examples, the air interface standard can be a Time Division Multiple Access (TDMA) air interface that supports a GSM or an advanced GSM protocol and air interface, or it may be a Code Division Multiple Access (CDMA) air interface standard. - The network operator includes or is coupled to a Streaming Content Server (SCS)60 that transmits streaming content towards the
mobile station 100, via theBSC 40 andBTS 50, which together may be considered to be equivalent to the data transmitter 1 shown in FIG. 1, while themobile station 100 may be considered to be equivalent to the data receiver orclient 3 of FIG. 1. TheSCS 60 operates in accordance with the foregoing description of the presently preferred embodiments of this invention to stream structured data towards themobile station 100, and themobile station 100, as the client, is responsive to the received stream to decode same and render the graphical content, as was illustrated above in reference to the example shown in FIGS. 3B-3G. - The
mobile station 100 typically includes a microcontrol unit (MCU) 120 having an output coupled to an input of adisplay 140 and an input coupled to an output of a keyboard orkeypad 160. Themobile station 100 may be a handheld radiotelephone, such as a cellular telephone or a personal communicator. Themobile station 100 could also be contained within a card or module that is connected during use to another device. For example, themobile station 10 could be contained within a PCMCIA or similar type of card or module that is installed during use within a portable data processor, such as a laptop or notebook computer, or even a computer that is wearable by the user. - The
MCU 120 is assumed to include or be coupled to some type of amemory 130, including a read-only memory (ROM) for storing an operating program, as well as a random access memory (RAM) for temporarily storing required data, scratchpad memory, received packet data, packet data to be transmitted, and the like. A separate, removable SIM (not shown) can be provided as well, the SIM storing, for example, a preferred Public Land Mobile Network (PLMN) list and other subscriber-related information. The ROM is assumed, for the purposes of this invention, to store a program enabling theMCU 120 to execute the software routines, layers and protocols required to implement thedata receiver 3 methods in accordance with these teachings, and may as well provide a suitable user interface (UI), via adisplay 140 andkeypad 160, with a user. The result of the rendering of the decoded structured data stream can be displayed on thedisplay 140, as was noted above in the discussion of FIG. 3G. - Although not shown, a microphone and speaker are typically provided for enabling the user to conduct voice calls in a conventional manner.
- The
mobile station 100 also contains a wireless section that includes a digital signal processor (DSP) 180, or equivalent high speed processor or logic, as well as a wireless transceiver that includes a transmitter 200 and areceiver 220, both of which are coupled to anantenna 240 for communication with the network operator. At least one local oscillator (LO) 260, such as a frequency synthesizer, is provided for tuning the transceiver. Data, such as digitized voice data, is transmitted and received through theantenna 240, and the structured data stream from the Streaming Content Server (SCS) 60 is also received through theantenna 240, and is thereafter processed by theDSP 180 and/or theMCU 120 to display the encoded animation or other video images to the user on thedisplay 160. - In view of the foregoing description, it should be appreciated that this invention overcomes the problems that are inherent in the prior art approaches discussed previously. For example, and with regard to the streaming of non-structured video and audio data, these formats are suited for streaming only non-structured binary data, such as video and audio, and not structured data.
- With regard to the Macromedia Flash™ technique, this approach uses a binary file format and cannot be used for generic structured data or XML. On the other hand, this invention enables the streaming of arbitrarily structured data. This invention can also be used to stream vector graphics using time-based animation. The example shown in FIG. 5 illustrates significant differences between streaming using time based animation versus the frame based approach. The boxes shown in FIG. 5A illustrate a streamed, time based animation content. The boxes of FIG. 5B illustrate a streamed, frame based animation. In this example, the frame-based approach requires streaming new coordinate values at each frame. On the other hand, time-based animation requires streaming only keyframes of the animation. The use of the teachings of this invention enables one to stream time-based animations, as well as any arbitrary time-based type of content.
- The approach taken in U.S. Pat. No. 6,057,854 relies on the presence of a client-server based system, without considering any streaming capabilities.
- The MPEG-7 BiM approach defines a compression and streaming method for structured data; but it does not define how the input data is partitioned into streamable subsets. This invention both addresses and solves this problem.
- The approach taken in WO 01/26373 A2 aims to solve the problem of transmitting a large XML tree, by subdividing it into a set of independent subtrees, which are transmitted separately. However, it does not provide a solution that allows the content creator to control the division of the XML tree. This creates problems in some types of structured content, e.g., when using vector graphic representations, because it does not take into account the semantics of the content. This invention, to the contrary, allows the content creator to control how the content is divided.
- The Push and Pull approach is not in and of itself a method to organize structured data, but instead represent a functionality of the HTML language and its transfer protocol HTTP. This invention solves a different problem than HTTP Push and Pull, as it enables the content creator to structure the data to be streamed.
- The SMIL technique has two elements ‘par’ (parallel) and ‘seq’ (sequential) that handle the timing and execution of SMIL content. However, these two elements do not introduce or define a streaming functionality. Put another way, SMIL enables the simple authoring of interactive audiovisual presentations, and is typically used for rich media/multimedia presentations that integrate streaming audio and video with images, text or any other media type. In all these cases the actual streaming is file format dependent and not related to the functionality of SMIL per se.
- As such, it should be appreciated that this invention provides significant advantages over the prior art techniques.
- The teachings of this invention can be applied to a workstation of a graphical content developer, such as one that forms a part of, or that is coupled to, the
SCS 60 of FIG. 4, or that is associated with thereceiver 3. In accordance with this invention the content developer may divide a pre-existing or a new temporally-based structured data graphical presentation, such as an animation, into smaller temporally-based scenes each having some temporal extension. These scenes are not, per se, equivalent to the conventional use of frames. In fact, some number of conventional frames can be incorporated within a single scene, thereby reducing the amount of structured data content that must be transmitted, processed and stored. These teachings enable a large and complex graphical presentation, such as an animation, to be efficiently transmitted to and displayed by a relatively small device, such as a cellular telephone, a personal communicator or a personal digital assistant (PDA). These teachings can be applied to any time-based graphical presentation, and in the specific, non-limiting embodiment disclosed above these teachings are applied to and used to extend the capabilities of SVG. - Thus, while these teachings have been particularly shown and described with respect to preferred embodiments thereof, it will be understood by those skilled in the art that changes in form and details may be made therein without departing from the scope of this invention.
Claims (30)
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/175,641 US7064760B2 (en) | 2002-06-19 | 2002-06-19 | Method and apparatus for extending structured content to support streaming |
PCT/IB2003/001900 WO2004001676A1 (en) | 2002-06-19 | 2003-05-12 | Method and apparatus for extending structured content to support streaming |
AU2003223075A AU2003223075A1 (en) | 2002-06-19 | 2003-05-12 | Method and apparatus for extending structured content to support streaming |
EP03719046A EP1527421A4 (en) | 2002-06-19 | 2003-05-12 | Method and apparatus for extending structured content to support streaming |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/175,641 US7064760B2 (en) | 2002-06-19 | 2002-06-19 | Method and apparatus for extending structured content to support streaming |
Publications (2)
Publication Number | Publication Date |
---|---|
US20040004619A1 true US20040004619A1 (en) | 2004-01-08 |
US7064760B2 US7064760B2 (en) | 2006-06-20 |
Family
ID=29999047
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/175,641 Expired - Lifetime US7064760B2 (en) | 2002-06-19 | 2002-06-19 | Method and apparatus for extending structured content to support streaming |
Country Status (4)
Country | Link |
---|---|
US (1) | US7064760B2 (en) |
EP (1) | EP1527421A4 (en) |
AU (1) | AU2003223075A1 (en) |
WO (1) | WO2004001676A1 (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030001868A1 (en) * | 2001-06-28 | 2003-01-02 | Ideaworks 3D Limited | Graphics compression |
US20030236903A1 (en) * | 2002-06-20 | 2003-12-25 | Koninklijke Philips Electronics N.V. | Method and apparatus for structured streaming of an XML document |
US20040064475A1 (en) * | 2002-09-27 | 2004-04-01 | International Business Machines Corporation | Methods for progressive encoding and multiplexing of web pages |
US20070083808A1 (en) * | 2005-10-07 | 2007-04-12 | Nokia Corporation | System and method for measuring SVG document similarity |
US20070097128A1 (en) * | 2005-11-02 | 2007-05-03 | Samsung Electronics Co., Ltd. | Apparatus and method for forming scene-based vector animation |
US20080091805A1 (en) * | 2006-10-12 | 2008-04-17 | Stephen Malaby | Method and apparatus for a fault resilient collaborative media serving array |
US20080120608A1 (en) * | 2006-11-17 | 2008-05-22 | Rohit Shetty | Generating a statistical tree for encoding/decoding an xml document |
US20080222504A1 (en) * | 2007-02-26 | 2008-09-11 | Nokia Corporation | Script-based system to perform dynamic updates to rich media content and services |
US20090307244A1 (en) * | 2008-06-08 | 2009-12-10 | International Business Machines Corporation | Encoding and decoding of xml document using statistical tree representing xsd defining xml document |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI363574B (en) * | 2003-04-07 | 2012-05-01 | Semiconductor Energy Lab | Electronic apparatus |
US20060025337A1 (en) | 2003-07-01 | 2006-02-02 | President And Fellows Of Harvard College | Sirtuin related therapeutics and diagnostics for neurodegenerative diseases |
US7681114B2 (en) | 2003-11-21 | 2010-03-16 | Bridgeborn, Llc | Method of authoring, deploying and using interactive, data-driven two or more dimensional content |
US8017634B2 (en) | 2003-12-29 | 2011-09-13 | President And Fellows Of Harvard College | Compositions for treating obesity and insulin resistance disorders |
US8645403B2 (en) * | 2006-02-03 | 2014-02-04 | Autodesk, Inc. | Database-managed rendering |
US8024356B2 (en) * | 2006-02-03 | 2011-09-20 | Autodesk, Inc. | Database-managed image processing |
US8434093B2 (en) | 2008-08-07 | 2013-04-30 | Code Systems Corporation | Method and system for virtualization of software applications |
US8776038B2 (en) | 2008-08-07 | 2014-07-08 | Code Systems Corporation | Method and system for configuration of virtualized software applications |
US20110116514A1 (en) * | 2009-11-13 | 2011-05-19 | Kirshenbaum Evan R | Communication system with nestable delimited streams |
US8954958B2 (en) | 2010-01-11 | 2015-02-10 | Code Systems Corporation | Method of configuring a virtual application |
US9104517B2 (en) | 2010-01-27 | 2015-08-11 | Code Systems Corporation | System for downloading and executing a virtual application |
US8959183B2 (en) * | 2010-01-27 | 2015-02-17 | Code Systems Corporation | System for downloading and executing a virtual application |
US9229748B2 (en) | 2010-01-29 | 2016-01-05 | Code Systems Corporation | Method and system for improving startup performance and interoperability of a virtual application |
US8516063B2 (en) | 2010-02-12 | 2013-08-20 | Mary Anne Fletcher | Mobile device streaming media application |
US8763009B2 (en) | 2010-04-17 | 2014-06-24 | Code Systems Corporation | Method of hosting a first application in a second application |
US9218359B2 (en) * | 2010-07-02 | 2015-12-22 | Code Systems Corporation | Method and system for profiling virtual application resource utilization patterns by executing virtualized application |
US9021015B2 (en) | 2010-10-18 | 2015-04-28 | Code Systems Corporation | Method and system for publishing virtual applications to a web server |
US9209976B2 (en) | 2010-10-29 | 2015-12-08 | Code Systems Corporation | Method and system for restricting execution of virtual applications to a managed process environment |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6057854A (en) * | 1997-03-07 | 2000-05-02 | Micrografx, Inc. | System and method of providing interactive vector graphics over a network |
US6584459B1 (en) * | 1998-10-08 | 2003-06-24 | International Business Machines Corporation | Database extender for storing, querying, and retrieving structured documents |
US6704024B2 (en) * | 2000-08-07 | 2004-03-09 | Zframe, Inc. | Visual content browsing using rasterized representations |
US6741242B1 (en) * | 2000-03-23 | 2004-05-25 | Famotik Kabushikikaisha | Multimedia documents integrating and displaying system |
US6785673B1 (en) * | 2000-02-09 | 2004-08-31 | At&T Corp. | Method for converting relational data into XML |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6966027B1 (en) | 1999-10-04 | 2005-11-15 | Koninklijke Philips Electronics N.V. | Method and apparatus for streaming XML content |
WO2001031497A1 (en) | 1999-10-22 | 2001-05-03 | Activesky, Inc. | An object oriented video system |
WO2002076058A2 (en) | 2001-03-21 | 2002-09-26 | Research In Motion Limited | Method and apparatus for providing content to media devices |
-
2002
- 2002-06-19 US US10/175,641 patent/US7064760B2/en not_active Expired - Lifetime
-
2003
- 2003-05-12 AU AU2003223075A patent/AU2003223075A1/en not_active Abandoned
- 2003-05-12 WO PCT/IB2003/001900 patent/WO2004001676A1/en not_active Application Discontinuation
- 2003-05-12 EP EP03719046A patent/EP1527421A4/en not_active Ceased
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6057854A (en) * | 1997-03-07 | 2000-05-02 | Micrografx, Inc. | System and method of providing interactive vector graphics over a network |
US6584459B1 (en) * | 1998-10-08 | 2003-06-24 | International Business Machines Corporation | Database extender for storing, querying, and retrieving structured documents |
US6785673B1 (en) * | 2000-02-09 | 2004-08-31 | At&T Corp. | Method for converting relational data into XML |
US6741242B1 (en) * | 2000-03-23 | 2004-05-25 | Famotik Kabushikikaisha | Multimedia documents integrating and displaying system |
US6704024B2 (en) * | 2000-08-07 | 2004-03-09 | Zframe, Inc. | Visual content browsing using rasterized representations |
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7307642B2 (en) * | 2001-06-28 | 2007-12-11 | Ideaworks 3D Ltd. | Graphics compression |
US20030001868A1 (en) * | 2001-06-28 | 2003-01-02 | Ideaworks 3D Limited | Graphics compression |
US20030236903A1 (en) * | 2002-06-20 | 2003-12-25 | Koninklijke Philips Electronics N.V. | Method and apparatus for structured streaming of an XML document |
US7251697B2 (en) * | 2002-06-20 | 2007-07-31 | Koninklijke Philips Electronics N.V. | Method and apparatus for structured streaming of an XML document |
US20040064475A1 (en) * | 2002-09-27 | 2004-04-01 | International Business Machines Corporation | Methods for progressive encoding and multiplexing of web pages |
US20070083808A1 (en) * | 2005-10-07 | 2007-04-12 | Nokia Corporation | System and method for measuring SVG document similarity |
US7403951B2 (en) * | 2005-10-07 | 2008-07-22 | Nokia Corporation | System and method for measuring SVG document similarity |
US20070097128A1 (en) * | 2005-11-02 | 2007-05-03 | Samsung Electronics Co., Ltd. | Apparatus and method for forming scene-based vector animation |
US8943218B2 (en) * | 2006-10-12 | 2015-01-27 | Concurrent Computer Corporation | Method and apparatus for a fault resilient collaborative media serving array |
US20080091805A1 (en) * | 2006-10-12 | 2008-04-17 | Stephen Malaby | Method and apparatus for a fault resilient collaborative media serving array |
US8972600B2 (en) | 2006-10-12 | 2015-03-03 | Concurrent Computer Corporation | Method and apparatus for a fault resilient collaborative media serving array |
US20090225649A1 (en) * | 2006-10-12 | 2009-09-10 | Stephen Malaby | Method and Apparatus for a Fault Resilient Collaborative Media Serving Array |
US20080120608A1 (en) * | 2006-11-17 | 2008-05-22 | Rohit Shetty | Generating a statistical tree for encoding/decoding an xml document |
US7886223B2 (en) * | 2006-11-17 | 2011-02-08 | International Business Machines Corporation | Generating a statistical tree for encoding/decoding an XML document |
US20080222504A1 (en) * | 2007-02-26 | 2008-09-11 | Nokia Corporation | Script-based system to perform dynamic updates to rich media content and services |
US7925643B2 (en) * | 2008-06-08 | 2011-04-12 | International Business Machines Corporation | Encoding and decoding of XML document using statistical tree representing XSD defining XML document |
US20090307244A1 (en) * | 2008-06-08 | 2009-12-10 | International Business Machines Corporation | Encoding and decoding of xml document using statistical tree representing xsd defining xml document |
Also Published As
Publication number | Publication date |
---|---|
EP1527421A4 (en) | 2005-08-31 |
WO2004001676A1 (en) | 2003-12-31 |
EP1527421A1 (en) | 2005-05-04 |
US7064760B2 (en) | 2006-06-20 |
AU2003223075A1 (en) | 2004-01-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7064760B2 (en) | Method and apparatus for extending structured content to support streaming | |
US6715126B1 (en) | Efficient streaming of synchronized web content from multiple sources | |
US20100138736A1 (en) | Delivering multimedia descriptions | |
US20080222504A1 (en) | Script-based system to perform dynamic updates to rich media content and services | |
KR100927978B1 (en) | How to embed SV content in an ISO-based media file format for progressive downloading and streaming of rich media content | |
US7865833B2 (en) | Data processing apparatus, data processing server, data processing system, method of controlling data processing apparatus, method of controlling data processing server, computer program, and computer readable storage medium | |
US20090157407A1 (en) | Methods, Apparatuses, and Computer Program Products for Semantic Media Conversion From Source Files to Audio/Video Files | |
EP1603046B1 (en) | Reception apparatus and information browsing method | |
JP2004533736A (en) | Method and apparatus for reformatting content for display on interactive television | |
KR20080083353A (en) | Extensions to rich media container format for use by mobile broadcast/multicast streaming servers | |
Dufourd et al. | An MPEG standard for rich media services | |
US6708337B2 (en) | Dynamic multimedia streaming using time-stamped remote instructions | |
CN106933853A (en) | A kind of files passe processing method and processing device | |
CN101617536B (en) | Method of transmitting at least one content representative of a service, from a server to a terminal, and corresponding device | |
KR20010028861A (en) | System and Method for Web Cataloging Dynamic Multimedia Using Java | |
Metso et al. | A content model for the mobile adaptation of multimedia information | |
JP2009508245A (en) | Method for transmitting multimedia content to a wireless communication terminal | |
US8230094B1 (en) | Media file format, system, and method | |
KR100597666B1 (en) | Method for browsing wireless internet document and terminal appratus implementing the same method | |
WO2001077894A1 (en) | Paged web protocol | |
Smith | Universal multimedia access | |
Davis et al. | Efficient delivery within the mpeg-21 framework | |
Koivisto | Multimedia Presentation and Transmission Standards and Their Support for Automatic Analysis, Conversion and Scalling: A Survey | |
AU2001268839B2 (en) | Delivering multimedia descriptions | |
Tseng et al. | Video personalization for usage environment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: NOKIA CORPORATION, FINLAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CAPIN, TOLGA;BALCISOY, SELIM;REEL/FRAME:013031/0928;SIGNING DATES FROM 20020617 TO 20020618 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
AS | Assignment |
Owner name: NOKIA TECHNOLOGIES OY, FINLAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NOKIA CORPORATION;REEL/FRAME:035600/0122 Effective date: 20150116 |
|
AS | Assignment |
Owner name: PROVENANCE ASSET GROUP LLC, CONNECTICUT Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NOKIA TECHNOLOGIES OY;NOKIA SOLUTIONS AND NETWORKS BV;ALCATEL LUCENT SAS;REEL/FRAME:043877/0001 Effective date: 20170912 Owner name: NOKIA USA INC., CALIFORNIA Free format text: SECURITY INTEREST;ASSIGNORS:PROVENANCE ASSET GROUP HOLDINGS, LLC;PROVENANCE ASSET GROUP LLC;REEL/FRAME:043879/0001 Effective date: 20170913 Owner name: CORTLAND CAPITAL MARKET SERVICES, LLC, ILLINOIS Free format text: SECURITY INTEREST;ASSIGNORS:PROVENANCE ASSET GROUP HOLDINGS, LLC;PROVENANCE ASSET GROUP, LLC;REEL/FRAME:043967/0001 Effective date: 20170913 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553) Year of fee payment: 12 |
|
AS | Assignment |
Owner name: NOKIA US HOLDINGS INC., NEW JERSEY Free format text: ASSIGNMENT AND ASSUMPTION AGREEMENT;ASSIGNOR:NOKIA USA INC.;REEL/FRAME:048370/0682 Effective date: 20181220 |
|
AS | Assignment |
Owner name: PROVENANCE ASSET GROUP LLC, CONNECTICUT Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CORTLAND CAPITAL MARKETS SERVICES LLC;REEL/FRAME:058983/0104 Effective date: 20211101 Owner name: PROVENANCE ASSET GROUP HOLDINGS LLC, CONNECTICUT Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CORTLAND CAPITAL MARKETS SERVICES LLC;REEL/FRAME:058983/0104 Effective date: 20211101 Owner name: PROVENANCE ASSET GROUP LLC, CONNECTICUT Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:NOKIA US HOLDINGS INC.;REEL/FRAME:058363/0723 Effective date: 20211129 Owner name: PROVENANCE ASSET GROUP HOLDINGS LLC, CONNECTICUT Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:NOKIA US HOLDINGS INC.;REEL/FRAME:058363/0723 Effective date: 20211129 |
|
AS | Assignment |
Owner name: RPX CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PROVENANCE ASSET GROUP LLC;REEL/FRAME:059352/0001 Effective date: 20211129 |
|
AS | Assignment |
Owner name: BARINGS FINANCE LLC, AS COLLATERAL AGENT, NORTH CAROLINA Free format text: PATENT SECURITY AGREEMENT;ASSIGNOR:RPX CORPORATION;REEL/FRAME:063429/0001 Effective date: 20220107 |