Glitch-free media streaming

1

GLITCH-FREE MEDIA STREAMING

BACKGROUND

Media content streaming services enable real-time play- 5 back of media content as it is downloading. Random seek capabilities in such streaming services facilitate rapid location of media clips related to user retrieval. Although such services are great for real-time media content playback, servers hosting Web pages that include media content do not 10 always provide content streaming service for real-time playback of hosted media content. Instead, many such Web sites support only HTTP-based Web page browsing and file downloading operations. As a result, to view a piece of media content from such a server, a user generally has to download 15 the media content from the server to a local machine. Then the user has to launch a local application to view the downloaded media content. Since media content files are typically very large, downloading an entire media file to view and/or search the content can be very time consuming for the user. However, 20 it is not realistic for all Web-based media content hosts to also provide media content streaming service.

SUMMARY

25

Glitch-free media streaming is provided by a client computing device ("client") independent of whether a remote computing device ("host") associated with media content provides real-time media content streaming services. Specifi- 3Q cally, the client computing device ("client") provides a user with real-time media content streaming services by establishing multiple connections to the host to read respective portions of the media content by sending byte-range transport protocol requests to the host. As the reading operations 35 progress, the client evaluates transmit rates over respective ones of the connections in view of a real-time media content playback rate to regulate data flow into a streaming media buffer and provide the user with a consistent high-quality (glitch-free) streaming media presentation.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determin- 45 ing the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the Figures, the left-most digit of a component reference 50 number identifies the particular Figure in which the component first appears.

FIG. 1 shows an exemplary system for glitch-free media streaming, according to one embodiment.

FIG. 2 shows exemplary evidence of data gap associated 55 with assignment of data segments that are too large to transport protocol connections for media streaming.

FIG. 3 shows exemplary evidence of data gap associated with assignment of data segments that are too small to trans port protocol connections for media streaming.

FIG. 4 shows exemplary transport protocol connection task assignment to prevent data gap during media streaming, according to one embodiment.

FIG. 5 shows exemplary bandwidth utilization for a chan- 65 nel of a single connection and for two channels of multiple connections, according to one embodiment.

FIG. 6 shows an exemplary procedure for glitch-free media streaming, according to one embodiment.

DETAILED DESCRIPTION

40

60

Overview

HTTP is based upon Transmission Control Protocol (TCP) protocol providing flow control, congestion control, and retransmission capabilities. Use of TCP for streaming typically results in long latency times and low data rates, which inevitably results in glitches during media content playback operations. This is because when an HTTP server receives a request for a media file hosted by a Web page, wherein the media file is being downloaded by a user for real-time playback, the HTTP server responds to the request identical to how the HTTP server would respond to a request to browse a Web page, perform a normal (non-streaming) file download, etc. The HTTP server does not take the real-time requirement of media streaming into consideration when responding to the request. Available bandwidth is fairly allocated to each connection.

In the scenario where a media content file is being downloaded for real-time playback, if available bandwidth is not high enough, it is difficult to maintain smooth media playback. Although enlarging client buffer size and pre-buffering additional data could be helpful, initial delay (latency) and buffer underflow (data rates) would likely result in poor playback quality. Additionally, although varying media frame playback speeds based on communication channel data throughput may avoid decoding buffer underflow conditions, such techniques may result in slow media playback speeds, causing undesired visual artifacts during media playback operations.

Systems and methods for glitch-free media streaming are described below in reference to FIGS. 1 -6. More particularly, the systems and methods provide a client-based solution to real-time glitch-free media content downloading and viewing. This solution provides a user with a high quality media content presentation without presentation latencies and/or dropped frames. To accomplish this, the client acquires media content from a host over multiple transport protocol connections at per-connection configured data rates that consider network bandwidth over time to provide consistent and controlled buffer management. The consistent and controlled buffer management regulates data input into the streaming media buffer to avoid buffer underflow and buffer overflow conditions, and thereby, provide for consistent real-time media content decoding and playback. This solution is independent of whether a Web server hosting media content of interest has access to real-time media content streaming and/ or seeking services to respond to a client request to stream a piece of media content. Although the systems and methods leverage transport protocols that support range queries, this solution is also independent of any changes to Web servers, Web pages, and/or transmission protocols.

These and other aspects of glitch-free media content streaming are now described in greater detail.

An Exemplary System

Although not required, glitch-free media streaming is described in the general context of computer-executable instructions (program modules) being executed by computing devices such as a general-purpose computer or a mobile handheld device. Program modules generally include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. While glitch-free media streaming is described in 3

the foregoing context, acts and operations described hereinafter may also be implemented in hardware.

FIG. 1 shows an exemplary system 100 for glitch-free media streaming, according to one embodiment. System 100 includes computing device ("client") 102 coupled across net- 5 work 104 to one or more remote computing devices 106. Network 104 may include any combination of a local area network (LAN) and a general wide area network (WAN) communication environments, such as those which are commonplace in offices, enterprise-wide computer networks, 10 intranets, and the Internet. Each client and remote computing device 102 and 106 include a respective processor coupled to a system memory. Each processor is configured to fetch and execute computer-program instructions stored in system memory to perform respective operations associated with 15 glitch-free media streaming.

For example, client 102 includes processor 108 coupled to system memory 110. System memory 110 includes program modules 112 and program data 114. In this implementation, program modules 112 include, for example, glitch-free media 20 content streaming module ("streaming module") 116 to provide a client-based solution to search and view a piece of downloaded media content 134 hosted by one or more remote computing devices ("hosts") 106. Downloaded media content 134 at any one time represents requested portions of media 25 content 118 from one or more hosts 106 (media content 118 may be hosted by more than one remote computing device 106). Program modules 112 also include, for example, media player 120 to present downloaded media content 134 to a user, and other program modules 122 such as an operating 30 system (OS), a browser to search for media content 118, and/or so on.

In one implementation, a search service is implemented on a respective remote computing device 106 (or on client 102) to respond to user media content search queries (e.g., input by 35 a user into a browser application, etc.). The search service indexes contents available on network 104 associated with key words. For instance, responsive to a user submitting a set of query key words, the search service evaluates the key words in view of the indexed content to returns a URL list of 40 remote computing device(s) 106 that host media content 118 related to the key words. For purposes of exemplary illustration, such a URL list is shown as a respective portion of "other program data" 126. For purposes of exemplary description, when a particular remote computing device 106 hosts a piece 45 of media content 118 of interest, the particular remote computing device 106 is frequently referred to hereinafter as a "host".

In one implementation, media content 118 is hosted by more than one remote computing device 106. When a user 50 expresses interest in a particular piece of media content in the URL list (e.g., via selection or some other modality), streaming module 116 establishes connections with one or multiple hosts 106 to simultaneously download one or more portions of media content 118 for glitch-free presentation to a user. 55 These connections are based on evaluated connection criteria between client 102 and respective ones of the hosts 106. This search scenario is only one example to identify one or more remote computing devices 106 that host a piece of media content 118. For example, such a URL list can also be 60 obtained from a network community, a body list in a messaging application, and/or so on.

Responsive to determining that a user is interested in a particular piece of media content 118 for a browsing application, streaming module 116 initializes multiple transport pro- 65 tocol connections 128 to one or more remote computing devices 106 hosting the particular piece of media content 118

4

(the hosts are identified in the URL list). As described in greater detail below in the section titled "Establishing Multiple Connections", the number of transport protocol connections 128 to host 106 initialized by client 102 is based on competing criteria. In this implementation, these criteria include, for example, two or more of network bandwidth, request overhead, connection overhead, real-time playback quality, buffer size, and decoding buffer management complexity. Connections 128 are logically shown in FIG. 1 as lines coupling client 102 across network 104 to remote computing device(s) 106.

In one implementation, transport protocol connections 128 are Hyper Text Transfer Protocol (HTTP) protocol connections. In another implementation, transport protocol connections 128 are based on a different transport protocol that also supports use of byte-range range requests to read specific portions of media content 118. Although system 100 can use any of multiple types of transport protocols, system 100 is described using connections based on HTTP for purposes of exemplary description and illustration.

Before requesting download of respective portions of media content 118 over respective ones of the connections 128, streaming module 116 determines for each connection 128 a corresponding appropriate data segment of media content 118 to read/download over the connection 128. The data segment indicates an offset in the media file and length (byte size) of the data segment, which is less than the total size of the media content 118. In one implementation, as soon as a data segment is determined for a particular connection 128, the particular connection can be utilized to access respective data segment portions of media content 118 (for streaming presentation to a user) before other ones of the connections 128 have been tasked to read respective data segments. Data segment sizes determinations for each connection 128 at any particular time are based on respective transmit times over the connection 128. To address fluctuating conditions of network 104 effecting data request and data transmission rates over time, streaming module 116 periodically re-evaluates data segment sizes forrespective ones ofthe connections 128. This periodicity is configurable. These and other aspects of tasking/configuring connections 128 to read data segment portions of media content 118 are discussed in greater detail below in the section titled "Available Connection Transmit Time".

To begin streaming media content 118 for glitch-free realtime presentation of corresponding downloaded media content 134 to a user (e.g., via a display device 132), streaming module 116 sends byte-range requests 130 to host 106 over respective ones of the established connections 128. For any particular connection 128, a byte range request 130 specifies a particular location (e.g., URI) of the content 118, a byteposition with respect to the beginning/start of the content 118, and the amount of content 118 (i.e. data segment size for the connection 128) subsequent to the byte-position of interest. Responsive to receipt of such a request 130, host 106 accesses and sends the requested portion of media content 118 to streaming module 116 in a corresponding response. Such a response is shown as an exemplary portion of "other program data" 126. This portion is communicated over the particular connection 128 that carried the request 130 begin addressed. Responsive to receiving responses to requests 130, streaming module 116 extracts the media content pay load from the response, and decodes the extracted payload using standard media content decoding techniques. Such decoded media content is shown at client 102 as a respective portion of "other program data" 126. Media player 120 presents the decoded media content to a user.

5

Additional exemplary aspects of establishing multiple connections 128 and determining appropriate data segment sizes for respective connections 128 are now described.

Establishing Multiple Connections 5

Prior to establishing multiple connections 128, streaming module 116 establishes a single connection 128 to host 106. Streaming module 116 obtains header data associated with media content 118 from host 106 using the single connection 10 128. In one implementation, such header information provides streaming module 116 with information to determine, for example, play duration and/or file size, bit-rate indicating a minimum download speed for real-time playback of downloaded media content 134 by media player 120, stream prop- 15 erties, and/or so on. The size and content of the header data is arbitrary, being based on the particular media file format associated with media content 118.

Although media content playback generally improves by increasing the number of connections 128, complexity of 20 buffer management and scheduling operations increases as the number of connections 128 increase. Additionally, because each request 130 sent over a connection 128 causes a certain amount of computational and data throughput overhead, media content 118 download delays can decrease 25 responsive to decreasing the number of requests 130 sent over a connection 128. Thus, streaming module 116 estimates a substantially optimal number of connections 128 to establish based on competing playback quality and complexity of download and decoding buffer management criteria. (For 30 purposes of exemplary illustration, an exemplary such download and decoding buffer(s) are shown as a respective portion of "other program data"). To this end, and in one implementation, streaming module 116 estimates current channel bandwidth RcA„0 and request 130 overhead T c. 35

More particularly, let Rplay represent bandwidth used for glitch-free playback of downloaded media content 134. Independent of overhead associated with multiple connections 128, an appropriate number of connections Nco„ for streaming module 116 to schedule in one implementation could be 40 provided as Rplay/Rchno. However, scheduling multiple connections 128 involves time consuming configurations of each connection 128 to read/download specific amounts of media content 118 from particular offset positions in the file. Assume that overhead of sending requests 130 is constant 45 when initializing connections 128. Thus, extending transmit time for each connection 128 being established to obtain longer data segments results in a smaller proportion of overhead compared with the transmit time. At the same time, a larger buffer is utilized to hold downloaded data segment; 50 otherwise the buffer would overflow. Therefore, maximum continuous transmit time for each connection is limited by client buffer (streaming buffer) size Bmax.

In view of the above, streaming module 116 identifies a smallest number of connections 128 to establish Nco„ accord- 55 ing to the following:

Bmax I Rplay ^ TreqD + Bmax / (Ncon ■ Rclmo) U)

and 60

> Rplay II Rplay ■Treqtl 'S (2)

65

This minimum number of connections Nco„ reduces overhead of sending requests 130. Additionally, Nco„ is selected such

6

that the connection overhead and time for associated buffer management operations still allow for real-time playback of downloaded media content 134.

Connection Transmit Rates

Streaming module 116 configures each connection 128 with a task to read a particular portion of media content 118. Each task is based on one or more of the following: (1) real-time parameters of each connection 128; (2) available transmit time for each connection 128; and, (3) received stream buffer fullness (status) at client 102. Real-time parameters for each connection 128 include, for example, data transmit rates in varying data throughput conditions and overhead of sending requests 130 over the connection. These parameters may differ across different connections 128. Streaming module 116 prioritizes use of connections 128 with faster transmit rates and lower request overhead for media content 118. For each connection i, two parameters (1) transmit rate for a connection 128 and (2) request 130 overhead time are respectively denoted by Rchni and T .. Streaming module 116 dynamically updates these parameters for at least a subset of connections 128 during media content streaming operations.

Streaming module 116 determines transmit rate for a connection 128 based on data length (d) and cost time (t) of recent r reading operations of a connection 128:

Streaming module 116 determines request 130 overhead time T . for a connections 128 using a previous request overhead time value (Treqi) an indication of time (t ) to send a last request 130 over the connection 128, and a weighting parameter a (e.g. a=0.5). In this implementation, the weighting parameter is an empirical parameter, which is analogous to a low-pass filter. The larger the weighting parameter, the less a current t effects a connection 128. Specifically, and in this implementation, T . is determined according to the following:

... (4)

As a special case, if distance AD,, between next reading position and current reading position with media content 118 is relatively small (e.g., smaller than a configurable threshold distance), streaming module 116 may not send new request 130 to move to the next reading position if time to process the request may be greater than it would take to continue streaming data from a current position to the next position without interruption. In this scenario, streaming module 116 adjusts request overhead time T ■ as follows:

... (5)

Exemplary Buffer Management

To ensure glitch-free media streaming, streaming module 116 ensures that downloaded media content 134 is available in the download buffer before it is decoded by media player 120 for presentation to a user. To this end, streaming module 116 ensures that there is no gap (missing information or buffer underflow) between received portion(s) of media content 118 and decoded data for playback in the client buffer (decoded