The following post and paper is from a guest contributor to the research blog. This post comes from Network Software Engineer, Jeongeun Julie Lee of Intel’s Communication Technology Lab where she researches WiMAX interoperability, WiMAX cross-layer MAC optimization, Traffic Modeling, and video over WiMAX. Her research interests include wireless communications, WiMAX MAC optimization, quality of service, sensor networks, and UWB application.

Given the wide use of HTTP traffic models to model user web browsing behavior, it is important that the model be representative of a large variety of traffic and be continually updated to reflect the constantly evolving nature of web content and the exponential growth in number of users. In this paper, we analyzed an extensive set of proxy web server logs to understand changes in network traffic patterns.

We found significant gaps in the methods previously proposed, specifically the major one being that it is almost impossible to detect a web request generated from a user click from one generated from various embedded scripts and frames. As a result, we modified the definition of a web request boundary. Due to the presence of large numbers of embedded objects from several different off-site sources, which cannot be traced back to the original request through following TCP/IP headers source addresses alone, newer heuristics need to be devised. We present our methodology for analyzing the squid proxy log in a way that preserves user privacy, and propose a new HTTP traffic model and traffic generator to represent current user web browsing behavior. Comparison of independent statistics from the trace and the model shows a fair match. You can read the entire paper here.