US20070283036A1

US20070283036A1 - System And Method For Providing A Web Page

Info

Publication number: US20070283036A1
Application number: US11/667,203
Authority: US
Inventors: Sujit Dey; Nishant Mittal
Original assignee: University of California
Current assignee: University of California
Priority date: 2004-11-17
Filing date: 2005-11-17
Publication date: 2007-12-06
Also published as: WO2006055769A2; EP1825388A2; WO2006055768A2; EP1825388A4; WO2006055769A3; US20080104231A1; JP2008521100A; WO2006055768A3; KR20070089155A; US8010655B2

Abstract

Method and system for providing a web page. Preferably, the method and system composes a web page based on real-time conditions. In an exemplary method, a web page including a plurality of data objects is received. For each of the plurality of data objects, content importance is evaluated. Available bytes that can be transmitted under current bandwidth of a communication channel to meet a desired latency goal are estimated, and the available transmission bytes are distributed among different ones of the plurality of data objects based on the evaluated content importance.

Description

PRIORITY CLAIM

This application claims priority of U.S. Provisional Patent Application Ser. No. 60/628,733, filed Nov. 17, 2004, and U.S. Provisional Application Ser. No. 60/628,729, filed Nov. 17, 2004.

TECHNICAL FIELD

A field of the invention is data communications. A more specific application of the invention is wireless data communications. Another more specific application of the invention is the transmission of World Wide Web (“web”) pages.

BACKGROUND ART

Internet access is offered in a wide variety of public and private settings through wireless connections. World Wide Web (“web”) usage, file transfer, multimedia applications, and other popular applications place high demand on wireless data connections. Users are becoming more dependent on such widespread access. Often, however, these experiences are unlike the experience provided in workplace and home environments, as extensive, unpredictable latency is a common characteristic of a wireless connection.
Conventional techniques attempt to make wireless web browsing faster by compressing (by a statically determined factor) all the images of a web page before transmitting over the wireless network. This reduces the volume of data to be transmitted, and reduces the web browsing latency by a constant factor, depending on the amount of (static) compression applied. However, this fails to address a fundamental problem in wireless web browsing—the unpredictable nature of wireless networks in terms of varying network bandwidth and error conditions—leading to unpredictable, and mostly slow, wireless web browsing speeds.
Additionally, static compression may result in images of poor quality being delivered to a user. Thus, a wireless service delivering such web pages may be less desirable for users, resulting in lost business for a wireless service provider and/or inconvenience for the user.
Other techniques for improving wireless data communication include trans-coding techniques that address the diversity and limitations of wireless handhelds. Trans-coding techniques attempt to re-format web content to map efficiently to different handhelds, in terms of their browser and display capabilities. However, trans-coding techniques do not address the variability of network conditions, and thereby cannot attempt to provide a fast and consistent wireless web browsing experience in the face of varying wireless network speeds.

DISCLOSURE OF THE INVENTION

Preferred embodiments of the present invention provide a method and system for providing a web page. In an exemplary method, a web page having a plurality of objects is received. For each of the plurality of objects, content importance is evaluated. Available bytes that can be transmitted under current bandwidth of a communication channel to meet a desired latency goal are estimated, and the available transmission bytes are distributed among different ones of the plurality of data objects based on the evaluated content importance. One or more of the data objects are dynamically object-specific shaped according to the allocated bytes for each object. A new web page is provided including the objects shaped by the dynamic object-specific shaping.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an exemplary wireless network architecture including a system for composing a web page based on real-time conditions, according to an embodiment of the present invention;
FIG. 2 shows exemplary system architecture for composing a web page based on real-time conditions, according to a preferred embodiment of the present invention; and
FIG. 3 shows an exemplary method for composing a web page based on real time conditions, according to an embodiment of the present invention.

BEST MODE OF CARRYING OUT THE INVENTION

Preferred methods and systems according to the present invention provide, among other things, improved delivery of a web page to a user. The web page requested by the user for delivery, for example, includes a collection of data objects such as text and image objects. Preferred embodiments improve delivery by estimating the available bytes that can be transmitted under current bandwidth of a communication channel to meet a desired latency goal (target latency), and allocating the available transmission bytes among different ones of the data objects based on particular evaluated criteria for each data object, thus selecting data objects for inclusion in the provided web page, and selecting a size for individual web page objects. To maximize quality of the provided web page, a preferred method selectively reduces the size of one or more of the data objects based on the allocated bytes for that object. The set of reduced data objects is dynamically composed (that is, in real time) to provide the new web page. In this way, the web page is modified based on real-time conditions.
Two preferred criteria for evaluating the objects are content importance for the objects and relationships among the objects. Upon a request for the web page, a preferred method evaluates each of the data objects for importance of the particular data object within the data collection (content importance) and the relationships among the objects (content semantics). Evaluating content importance, for example, may be done by employing a learning algorithm that evaluates objects for content importance, or by receiving and evaluating content parameters that are provided (e.g., by a third party). Consideration of these criteria allows targeted object selection and/or object size reduction. This reduces the required overall bandwidth for a requested web page and permits transmission meeting the target latency, while maximizing the quality of the web page.
Given a request for a web page, for example, preferred methods and systems provide a dynamic technique, referred to herein as web shaping, that recomposes the web pages to be delivered to a user in real time. A device or system performing such methods is referred to herein as a web shaper. In a preferred embodiment, a web shaper serves web requests within a target latency while maximizing web page quality, under a current network condition. The web shaper meets the target latency by dynamically shaping the content of the web page to approach or meet the maximum amount of information that can be sent under current network bandwidth available without violating the target latency. While transmitting the required amount of information, the preferred web shaper attempts to maximize the visual quality of the web page, which generally refers to the sum of quality of all objects present weighted by their importance towards the web page.
Visual quality is maximized in exemplary embodiments by selecting a subset of data objects in the web page guided by their content importance and/or content semantics and allocating the available bytes to each selected data object. Particularly, by considering the importance of each object, and the relationships between the objects, a preferred web shaper enables generation of web pages with highest content quality possible under specified constraints.
A preferred method estimates available bytes that can be transmitted under a current bandwidth of a communication channel to meet the target latency. For example, a preferred method receives or provides an estimated network data rate, and uses this information with a provided desired latency constraint to determine a total number of available transmission bytes. The available transmission bytes are distributed among the different data objects, preferably according to evaluated content importance and/or content semantics criteria. Recomposing the web page, according to embodiments of the invention, includes selecting objects for inclusion in a provided web page and dynamic object-specific shaping one or more of the data objects within the web page to meet the allocated bytes for each object.
A web shaper in accordance with preferred embodiments of the invention can thus dynamically compose web pages, depending on the current, experienced network bandwidth, to provide a consistent, pre-specified web browsing latency. Instead of just compressing images according to a static factor, a preferred method analyzes the web content real-time; and re-composes a new web page, respectful of the priorities, semantics, and relationships of all the objects in the web page, and with an aim to maximize the quality of the delivered web page, given the network bandwidth estimates and pre-selected latency constraint. Initial studies show up to 10× improvement in wireless web browsing speeds over any wireless network, and up to 4× improvement of web browsing speeds compared to existing, static image compression based techniques.
For a wireless network, embodiments of a wireless web shaper provide significant value propositions to both content providers and wireless carriers, as such embodiments can scale web content to fit to any network and network conditions (from original content to text-only content, and any version in between) in real-time. With a preferred web-shaper, content providers need only maintain one version of their web content to support diverse wireless network access technologies and wireless handhelds.
Additionally, the web shaper can enable wireless carriers to offer wireless web services at lower data rates, with higher revenues, and content differentiations according to different Service Level Agreements (SLA). Because preferred embodiments of the invention can best-fit web content into any kind of wireless pipe, including thin/noisy pipes, embodiments of the invention can tremendously boost data ARPU (average revenue per unit or user) of carriers, since they can significantly increase the number of data subscribers in a cell, while ensuring an acceptable level of data user satisfaction. Also, using a preferred web shaping method, carriers are able to offer differentiated services, such as by scaling content depending on SLAs.
Referring now to the drawings, an exemplary wireless network 10 is shown in FIG. 1, an embodiment of a preferred web shaper 12 is shown in FIG. 2, and an exemplary web shaping method is shown in FIG. 3. Generally, the web shaper 12 receives web page content 14, including a collection of data objects, in response to a request for a data communication by a user. A user, as non-limiting examples, may include a wireless GPRS client 16, an EDGE client 18, a W-CDMA client, 20, or an 802.11x client 22. The client 16, 18, 20, 22 accesses the internet 24 via a carrier gateway 26. A base station controller 28 preferably also is provided as part of a carrier infrastructure. As a general example, a preferred embodiment may be applied to a 3rd Generation (3G) wireless data network.
The web shaper 12 may be embodied, for example, as server software residing on the carrier gateway 26. Alternatively or additionally, the server software may reside on a content provider 30. The content provider 30, along with a content aggregator 32 and/or a wireless web host 34, may provide web pages to the client 16, 18, 20, 22 over the internet 24. However, a benefit of preferred embodiments of the present invention is implementation flexibility, in that the content may be a web page received directly from the content provider 30, aggregated content from the content aggregator 32, or content already modified for wireless use from the wireless web host 34.
The web shaper 12 may act as a proxy server that receives content in response to a request to the server, and customizes the content. In the preferred method illustrated in FIGS. 2 and 3, in response to a request from the client 16, 18, 20, 22, content 14 is furnished including web page content (step 100). This content, for example, may be in the form of an HTML page that is parsed into relevant objects (e.g., images, text, JavaScript, CSS file, etc.) The content 14 may be from a content server such as the content provider 30, content aggregator 32, and/or the wireless web host 34, and it includes a plurality of data objects. If the web page is not updated on the content server, a cache (not shown) local to the web shaper 12 may store the retrieved content for reading by the web shaper.
The content 14 received by the web shaper 12 includes, for example, the collection of objects forming the web page, including image and HTML objects. Content parameters 36 also are supplied to the web shaper (step 102), preferably from the content provider 30. These parameters 36 may include, for example, supplied priorities or importance of particular objects, which may allow the evaluation of content importance performed by the web shaper 12 to be customized by the content provider 30 or other party.
For determining the bytes (bandwidth) available for the client 16, 18, 20, 22 under particular network conditions, a network data rate 38 and a desired latency 40 preferably are supplied (step 102) to the web shaper as inputs. By providing the data rate and the desired latency, the web shaper 12 can determine the number of bytes that are available for transmitting a web page under dynamic network conditions without violating the latency constraint.
The desired latency 40 preferably is a predetermined value supplied by, for example, the content provider 30, the wireless web host 34, the carrier 26, etc. Desired latency generally refers to the amount of time acceptable to the client 16, 18, 20, 22 for waiting for a requested web page to be received. This amount of time may be an initial default value, a selected value, or a value determined in one or more of various ways (e.g., a desired latency may be chosen based on a particular wireless service plan, or based on a particular content provider). The latency, for example, may be predetermined so that the provided or estimated available bytes are determined to reduce the likelihood of slow or frequently interrupted data communication.
The network data rate 38 is provided to the web shaper in real time to provide a web page that is customized dynamically depending on changing network conditions. The data rate 38 or network speed, which may be expressed, for example, in number of bits/second, may be determined in any of various ways. Exemplary methods for determining the data rate 38 include a web sniffer installed onto the carrier gateway 26, the content provider 30, or the wireless web host 34. However, proxy servers, if used, may break up the connection. Another method of estimating the data rate uses Ethereal or other suitable client-based software that monitors the packets that are downloaded to the client 16, 18, 20, 22 and measure the experienced data rate as seen by the client. Those of ordinary skill in the art will appreciate that there are various ways to estimate the network data rate 38, and any suitable method may be used for supplying the data rate to the web shaper 12.
In a preferred embodiment, the data rate 38 is estimated by inserting a web scripting language (e.g., JavaScript or Perl) into a web page that provides a time stamp for when data is first received by the client 16, 18, 20, 22, and when the data was fully received by the client. The web page may be a page generated by the content provider 30, when time information is desired without sending a large amount of data to the client. Alternatively, the web page may be a web page generated solely to test the data rate, to estimate the bandwidth availability when no a-priori knowledge of current network condition is available, or when a certain amount of time has elapsed. If a new web page is generated, the selection of techniques for determining a data rate may be dynamically customized based on the application requirements as well as the network variability observed. Further, the selection may be made conscious of the tradeoffs involved between the accuracy of the bandwidth estimation and the potential overhead in terms of time consumed and additional traffic used. Under specified limits of overhead, a preferred method attempts to achieve best-effort network bandwidth estimation. This method preferably avoids an additional software download by the client 16, 18, 20, 22 (beyond the web browser).
As shown in FIGS. 2 and 3, the provided network data rate 38 (e.g., in bytes/second) is simply multiplied by the desired latency (e.g., in seconds) to compute an estimate of the available bytes 41 that can be transmitted under current bandwidth in a communication channel while meeting the desired latency (step 104). The available bytes 41 preferably are then allocated among various objects of the web page, and may be used to determine if the final modified web page will be more text-centric (for fewer available bytes) or can include more images (for more available bytes).
In a preferred embodiment, a web scripting language, such as JavaScript, is used to determine device information about the client 16, 18, 20, 22, which is also sent to the web shaper 12 (step 102). The scripting language is sent from the web shaper 12 to the client, and it is implemented using the client's web browser to send back the device information. Such device information includes the display size 42 of the client device and the device capabilities 44. The display size 42 and device capabilities are useful to optimize the final customized web page for the particular client 16, 18, 20, 22.
The device information allows optimization of the web page for a particular client 16, 18, 20, 22. To maximize quality of the customized content, the content 14 is passed through one or more analysis steps, including a content analyzer 50 and a content semantics (relationships) analyzer 52. Generally, an exemplary content analyzer 50 evaluates (step 106) a relative importance of each object in the web page (i.e., content importance) to determine priorities among the objects.
In an exemplary embodiment, the content analyzer 50 takes the original web page as input, parses the web page to provide the individual objects, and analyzes the content (i.e., each data object) to find out relevant application-level data properties, particularly content importance. Preferably, the content analyzer analyzes each of the individual data objects making up the web page for aspects of content importance for the object, and assigns a priority expressed as a number, which is incorporated into the markup language for the web page. The content analyzer 50 preferably also provides in the markup language a size, in bytes, of each object on the web page.
For example, the content analyzer 50 may use pre-characterized information about the web page data objects to determine importance. In exemplary embodiments, coding for a web page (e.g., markup language) may include data characterizing the relative importance of one or more data objects. More particularly, the content importance can be explicitly provided by the content service provider as a tag along with the content.
Alternatively, embodiments of the content analyzer 50 may use a learning algorithm to analyze content, and estimate the importance of different objects of the content. In an exemplary embodiment, the content analyzer 50 identifies a relative importance of each object. In a preferred embodiment, the content analyzer 50 employs a heuristic algorithm that considers several characteristics of individual data objects, such as physical location of the object within the web page, display size of the object, HTML formatting tags, etc.
The intuition behind the heuristic algorithm for the content analyzer 50 is to identify the visual impact each individual object creates to the user as compared to other objects on the page. Different objects contribute different levels of importance to the overall content quality; for example, an image used for navigational purpose in a web page may have lower importance than other images. Further, a central headline or image may have more importance than other text or images.
The output of the content analyzer 50 is a web page having priorities assigned to each of the HTML and image objects. The supplied (or default) content parameters 36 are combined (preferably, XOR'ed) 51 with the output of the content analyzer 50 to incorporate any customization of the priorities that is provided by the web shaper 12 or other parties. Thus, evaluating content importance may be performed using the content analyzer and/or using the supplied content parameters. The output is fed into the content semantics (relationship) module 52, for evaluating (step 108) a relationship of each object among the various objects in the web page, and modifying the determined priorities accordingly to determine the analyzed content 54.
Preferably, the content semantics step 52 considers the relationships between all data objects on the web page to evaluate the importance of each object, and modifies the priorities determined by the content analyzer 50 and/or the content parameters 36 based on relationships among the individual web page objects. This is done to preserve the semantics of the information in the recomposed web page. For example, in a news webpage having a central article including a top image, a headline, and accompanying text, it is preferred that the top image, headline, and text be kept together in a final web page. Thus, the content semantics step 52 would raise the priority of the headline and text that are related to the top image.
Additionally, in a preferred embodiment, the available bytes 41 amount is supplied to the content semantics module 52 for modifying priorities of objects based on the bandwidth. The content semantics step 52 preferably adjusts for the available bytes 41 by modifying the importance for the image objects in the web page. For example, if the number of available bytes 41 is too low, the content semantics step can consider this information and reduce the number of images by lowering the priority of images. Similarly, if the available bandwidth is high, several images can be supplied by raising the priority of images.
As a result, the content semantics step 52 outputs (step 110) analyzed content 54 in server memory, preferably in the form of a modified web page with priority information (e.g., a number representing modified priority) assigned in the markup language to each object in the web page. Additionally, as stated above, a size of each object is provided (e.g., in bytes) in the modified web page.
Based on the analyzed content 54, the available bytes 41, and preferably the display size 42 and device capabilities 44, a content/quality selector 56 allocates (step 112) available transmission bytes among different objects in the data collection (e.g. data objects in the web page). The content/quality selector 56 preferably uses a priority-based algorithm, with the modified priority from the content semantics step 52 used as the priority value to allocate the available transmission bytes among the different data objects of the web page. Each data object thus is given a target size, for example, in allocated bytes.
In a preferred embodiment, the content/quality selector 56 allocates bytes among each of the collection of data objects making up the web page. Preferably, this is done by first arranging all of the objects (step 114) in the web page supplied by the analyzed content 54 in a sorting order, most preferably ascending order, determined by priority and size for each object. For example, the sorting order may be determined by a ratio of object priority to object size. In this way, each object, which is demanding certain bytes, is provided a priority for each byte in that object, and the objects are ranked accordingly.
Objects are selected (step 116) from the sorted list one by one. In a preferred embodiment, the first object selected from among the available objects is the object having the lowest priority-per-byte ratio. For the selected object, available transmission bytes are assigned (step 118), preferably according to the formula [weighted importance divided by total importance times available bytes]. If the number of assigned bytes for the selected object is less then the minimum number of bytes (step 120) needed by the object, then it is decided to delete (step 122) the object. The bytes previously assigned to this object are returned to the available bytes pool so that they can be utilized by other objects on the page.
If the number of assigned bytes is greater then the minimum needed, a compression ratio is computed (step 124), preferably as the ratio of assigned bytes to the total bytes needed by the object when uncompressed. Given the computed compression ratio, if the selected object is an image (step 126) then the image shaper 60 is launched (step 128) to compress the image according to the compression ratio determined in the last step. If any bytes are left over after the image shaper 60 is done, these bytes are returned to the available bytes pool 62 for use by other objects. If there are more objects in the sorted list (step 130) then the next object is selected in the ascending order (step 116) and the above process is repeated for this selected object.
In a preferred embodiment, to account for the display size 42 and device capabilities 44, images that are selected for forwarding to the client 16, 18, 20, 22 may be modified (e.g., by the image shaper 60) to meet the device requirements. For example, for a device having only a monochrome or gray display, selected color images may be made monochrome or gray. Images too large for a particular device may be resized. Additional bytes are also placed into the unused bytes pool 62. This provides final, customized content 64 including a web page having the shaped data objects to which transmission bytes have been allocated.
Once all objects in the sorted list are considered (step 130), the HTML page is shaped (step 132). For example, if the object is HTML content (HTML body text), the HTML shaper 58 recreates or modifies the HTML (step 132) by removing objects selected for removal or truncating objects according to the selected compression ratio so that the page fits within the allocated bytes according to the content/quality selector 16.
The web shaper 12 thus provides customized content 64 (step 132) in the form of a new data collection, such as a new HTML web page and images. In this new web page, the HTML now includes objects selected for inclusion, selectively compressed according to a ratio based on real-time network conditions. For the web page images, a compression level may be provided in the markup. Preferably, the markup language requests the shaped (compressed) images, which are stored in the server on which the web shaper 40 resides. The new web page 64 and shaped images are delivered to the client 16, 18, 20, 22.
Preferred embodiment methods do not require any additional software at the client side. In addition, embodiments can exploit the programmability currently available in all web browsers, and hence can support a diverse variety of clients without any modification. Network monitoring in accordance with the invention is configurable to provide optimal network condition estimation under specified monitoring overhead in terms of the time consumed or the overhead traffic. Advantageously, embodiments of the invention exploit the passive network-related information observed from the past communication to predict the network condition for future, thereby reduces the monitoring overhead significantly.
Preferred embodiments of the present invention can potentially provide improved web service for devices that do not have a predictable, broadband connection to the Internet. Embodiments of the invention can be used with existing or yet-to-be created wireless access technology for web access.
While various embodiments of the present invention have been shown and described, it should be understood that other modifications, substitutions, and alternatives are apparent to one of ordinary skill in the art. Such modifications, substitutions, and alternatives can be made without departing from the spirit and scope of the invention, as set forth in the appended claims.
Various features of the invention are set forth in the appended claims.

Claims

1. A method for providing a web page, the method comprising steps of:

receiving a requested web page including a plurality of data objects;

for each of the plurality of data objects, evaluating content importance;

estimating available bytes that can be transmitted under current bandwidth of a communication channel to meet a desired latency goal;

distributing the available transmission bytes among different ones of the plurality of data objects based on the evaluated content importance;

dynamic object-specific shaping the plurality of data objects to meet the distributed bytes for each object;

composing a modified web page including the objects shaped by said dynamic object-specific shaping, to be delivered to the communication channel.

2. The method of claim 1, further comprising:

selecting at least one of the plurality of data objects for inclusion or deletion in the modified web page based on the allocated transmission bytes;

wherein said dynamic object-specific shaping is performed on the selected data object.

3. The method of claim 2, further comprising:

determining a ratio of evaluated content importance to size for each of the plurality of data objects;

choosing an object having the lowest ratio of evaluated content importance to size for said selecting for inclusion or deletion.

4. The method of claim 1, wherein said dynamic object-specific shaping comprises image compressing an object if the object is an image, and truncating an object if the object is a text-based object.

5. The method of claim 1, wherein said composing occurs in real time.

6. The method of claim 1 further comprising:

for each of the plurality of data objects, evaluating a relationship among the respective data objects;

wherein said allocating is further based on the evaluated relationship.

7. The method of claim 6, further comprising:

modifying the evaluated content importance based on the evaluated relationship.

8. The method of claim 1, further comprising:

modifying the evaluated content importance for image objects based on the estimated available bytes.

9. The method of claim 1, further comprising:

arranging the plurality of data objects according to the determined ratio;

choosing one of the plurality of data objects having the lowest ratio of evaluated content importance to size;

selecting the chosen data object for inclusion or deletion in the modified web page based on the allocated transmission bytes.

10. The method of claim 9, further comprising:

if the chosen data object is selected for deletion, adding bytes allocated to the chosen data object to the available transmission bytes;

if the chosen data object is selected for inclusion, performing said dynamic object-specific shaping on the chosen data object, and if additional bytes remain, adding the remaining bytes to the available transmission bytes;

re-allocating the transmission bytes among the remaining data objects.

11. A method for providing a data communication given a data rate, the method comprising:

receiving a request for communication of a data collection including a plurality of data objects;

retrieving the data objects;

evaluating each of the retrieved data objects for content importance;

estimating available bytes using the data rate and a desired latency;

reducing each of the retrieved data objects, respectively, by an amount that is based at least on the object's evaluated content importance and estimated available bytes.

12. The method of claim 11, wherein said evaluating comprises evaluating at least one of relative importance of each of the data objects, and evaluating relationships among each of the data objects.

13. The method of claim 12, wherein the amount of said reducing is further based on at least one of size of a client display and capabilities of a client.

14. An apparatus for modifying a web page in real time from a received request, the apparatus comprising:

a memory storing a web page including a collection of data objects that is retrieved based on said request;

a content importance evaluator module for evaluating at least content importance for each of the data objects stored in said cache;

a content/quality selector for distributing transmission bytes to each of the data objects based on the evaluated content importance and a predetermined overall available bandwidth; and

a shaper for dynamically object-specific shaping the data objects stored in said memory according to the distributed transmission bytes and providing the new web page including the dynamically object-specific shaped data objects.

15. The apparatus of claim 14, further comprising:

a content semantics module for evaluating relationships among the data objects, and modifying the evaluated content importance based on the evaluated relationships.

16. The apparatus of claim 15, wherein said content semantics module further modifies the evaluated content importance based on the estimated available bandwidth.

17. The apparatus of claim 14, wherein said shaper comprises:

an HTML shaper for deleting one or more objects;

an image shaper for compressing images by an amount depending on the allocated transmission bytes.

18. The method of claim 1 wherein said evaluating content importance comprises at least one of analyzing one or more of the data objects for content importance and receiving content parameters providing content importance for the one or more of the data objects.