XHR WDIO Helper class

It is nowadays easily possible with tools as Playwright and cypress to measure the performance or do performance test and monitoring on UI automation in some way, relying on the devtools. For selenium I created for a project of mine also a custom option, the XHR WDIO helper class(am a big fan of wdio project)

Before we dive into details first will explain what XHR means

XHR stands for XMLHttpRequest, which is a JavaScript API provided by web browsers that allows web pages to make asynchronous HTTP requests to the server. This means that a web page can request, retrieve, and send data to a server in the background, without having to perform a full page refresh. This capability is fundamental to the development of dynamic, responsive web applications.

Origins and Evolution

The XMLHttpRequest object was originally designed by Microsoft and adopted by Mozilla, Apple, and Google, eventually becoming a standard tool in all modern web browsers. Its development and widespread adoption were pivotal in the evolution of web applications from static pages to dynamic, interactive experiences, leading to what is now known as Ajax (Asynchronous JavaScript and XML) programming.

How XHR Works

• Making Requests: With XHR, a web application can send a request to the server and process the response within the same page. This request can be made to the same server that served the web page or, subject to certain restrictions for security reasons, to a different server.
• Handling Responses: The server response can be processed as soon as it’s received, allowing the web page to update dynamically. Responses can be handled in various formats, including plain text, XML, JSON, and HTML.

In devtools you can measure your xml http request easily. Nowadays in Selenium grid 4 you have support from devtools and for this I thought I need to have some functionality to measure and address the performance and functional monitoring

Why: The Purpose of the XHRhelper Class

• Identify Performance Bottlenecks and functional Behavioral Inconsistencies: In modern web development, ensuring that applications perform efficiently and behave consistently across different environments is crucial. The asynchronous nature of XHR requests, which are fundamental to dynamic web applications, can introduce performance issues and unpredictable behavior that are not always apparent during standard testing.
• Enhance Quality Assurance at Scale: As applications grow in complexity, testing them manually or with basic automated tools becomes insufficient. There’s a need for more sophisticated testing methods that can simulate real-world conditions, identify potential problems early, and ensure that the application meets its performance and reliability standards.

How: Leveraging DevTools and Selenium (Grid)

• Integration with Browser DevTools: The XHRhelper class utilizes the capabilities of browser DevTools to monitor and capture detailed information about XHR requests and responses. This approach allows for precise measurement of request durations, analysis of status codes, and inspection of headers and payloads, providing deep insights into the inner workings of web applications.
• Scalability through Selenium (Grid) and devtools By operating within a Selenium (Grid) and devtools environment, the XHRhelper class enables tests to be executed across a wide array of browsers and operating systems simultaneously. This distributed testing framework ensures that the application’s performance and behavior are thoroughly evaluated under diverse conditions, reflecting a more accurate range of user experiences.

What: The Functionality and Benefits of XHRhelper

• Automated Monitoring and Analysis: The class automates the process of monitoring XHR requests, capturing a wealth of data about each request, and analyzing this data to uncover issues. It tracks requests in real-time, identifies those that do not complete within expected time frames, and logs detailed information for further analysis.
• Customizable and Controlled Testing Environment: Through methods like enableMonitoring, disableMonitoring, setFilterPattern, and waitForXHRs, testers can tailor the monitoring process to focus on specific areas of interest, manage the scope of the test, and control the timing of test execution. This level of control is essential for pinpointing issues and optimizing performance.
• Facilitation of Performance Optimization: By providing detailed insights into the performance characteristics of XHR requests, the XHRhelper class enables developers to identify inefficiencies and optimize their code. This can lead to faster page loads, more responsive applications, and a better overall user experience. In summary, the XHRhelper class serves as a sophisticated tool for enhancing the quality assurance process of web applications. By leveraging the capabilities of browser DevTools and Selenium Grid, it allows for comprehensive monitoring and analysis of XHR requests at scale, offering valuable insights that drive performance optimization and ensure consistent behavior across different platforms.

import { browser } from '@wdio/globals';

class XHRhelper {
    constructor() {
        this.xhrData = [];
        this.pendingRequests = {};
        this.xhrCount = 0;
        this.lastXHRInitiationTime = 0;
        this.isXHRMonitoring = true;
        this.filterPattern = null; // No filter by default
        this.unfinishedXHRs = [];
    }
   

    enable() {
        browser.on('Network.requestWillBeSent', (params) => {
            if (params.type === 'XHR' && this.isXHRMonitoring && this.matchesFilter(params.request.url)) {
               // console.log('Network.requestWillBeSent detected!', params);
               
                this.xhrCount++;
                this.lastXHRInitiationTime = new Date().getTime();
                this.pendingRequests[params.requestId] = {
                    timestamp: params.timestamp,
                    headers: params.request.headers
                };

                // Track the initiation of an XHR request
                this.unfinishedXHRs.push({
                    requestId: params.requestId,
                    url: params.request.url,
                    finished: false,
                    headers: params.request.headers
                });
            }
        });

        browser.on('Network.responseReceived', (params) => {
            if (params.type === 'XHR' && this.isXHRMonitoring && this.pendingRequests[params.requestId]) {
                //console.log('Network.responseReceived detected!', params);
                this.xhrCount--;
                const requestInfo = this.pendingRequests[params.requestId];
                const duration = params.timestamp - requestInfo.timestamp;
                this.xhrData.push({
                    url: params.response.url,
                    statusCode: params.response.status,
                    duration: duration * 1000,
                    requestHeaders: requestInfo.headers
                });

                // Mark XHR as finished
                const xhrIndex = this.unfinishedXHRs.findIndex(x => x.requestId === params.requestId);
                if (xhrIndex !== -1) {
                    this.unfinishedXHRs[xhrIndex].finished = true;
                }

                delete this.pendingRequests[params.requestId];
            }
        });
    }

    disableMonitoring() {
        console.log('Disabling XHR monitoring...');
        this.isXHRMonitoring = false;
    }

    enableMonitoring() {
        console.log('Enabling XHR monitoring...');
        this.isXHRMonitoring = true;
    }

    setFilterPattern(pattern) {
        console.log('Setting filter pattern...');
        this.filterPattern = pattern;
    }

    matchesFilter(url) {
        return !this.filterPattern || url.includes(this.filterPattern);
    }

    async waitForXHRs(gracePeriod = 3000) {

    try {
        await browser.waitUntil(() => {
            const now = new Date().getTime();
            const isWaitingOver = this.xhrCount === 0 && (now - this.lastXHRInitiationTime > gracePeriod);
            return isWaitingOver;
        }, {
            timeout: 30000,
            timeoutMsg: 'XHRs did not finish within the expected time.'
        });
    }
    catch (error) {
    console.log('Continuing despite the error:', error.message);
    }

    this.unfinishedXHRs.forEach(xhr => {
        console.log({
            url: xhr.url,
            statusCode: xhr.finished ? 'Finished' : 'Not Finished',
            requestHeaders: xhr.headers
        });
    });
   
        console.log('Continuing with the script. All XHRs, finished or not, are logged.');
    }

    getXHRErrors() {
        // Implement this method based on your requirements
        // For example, filter xhrData for entries with non-200 status codes
    }

   async resetXHRData() {
        this.xhrData = [];
        this.unfinishedXHRs = [];
    }

    async waitForAllXHRsToComplete(gracePeriod) {
        const timeout = 30000; // maximum time to wait for XHRs to complete
        try {
            await browser.waitUntil(() => {
                const now = new Date().getTime();
                const isWaitingOver = this.xhrCount === 0 || (now - this.lastXHRInitiationTime > gracePeriod);
                return isWaitingOver;
            }, {
                timeout: timeout,
                timeoutMsg: 'XHRs did not finish within the expected time.'
            });
        } catch (error) {
            throw new Error('Timeout waiting for XHRs to complete');
        }
    }

    getXHRData() {
        return this.xhrData;
    }
}

export default new XHRhelper();

The provided code defines a JavaScript class named XHRhelper designed to assist with monitoring and handling XML HTTP Requests (XHRs) within web applications, particularly when using the WebDriverIO (WDIO) testing framework.

Usage

This helper class is designed to be imported and used in a testing environment where WebDriverIO is employed to automate browser actions. It provides detailed monitoring and logging capabilities for XHR requests, which can be invaluable for testing the asynchronous behavior of web applications, ensuring that all XHR requests complete as expected, and identifying potential issues with request handling.

To use XHRhelper, one would typically instantiate the class (or use the exported instance), call enable() to start monitoring XHR requests, and then interact with the web application under test. The other methods provide additional control and insights, such as enabling/disabling monitoring, setting a filter pattern for specific requests, waiting for requests to complete, and retrieving collected XHR data.