Implementing Newman for Running Postman Tests in CI/CD Pipelines

Introduction

Newman is a command-line collection runner for Postman, allowing you to run and test your Postman collections directly from the command line. It integrates seamlessly with CI/CD pipelines, enabling automatic testing of APIs. In this blog post, we’ll guide you through setting up a test case in Postman, running it with Newman, and integrating it into a CI/CD pipeline.

1. Creating a Test Case in Postman

Step 1: Open Postman and create a new collection. Step 2: Add a new request to the collection and set up the API request you want to test. Step 3: Under the “Tests” tab, write tests for your API request using JavaScript. Here is an example testing my bonus caculator:

pm.test("Status code is 200", function () {
  pm.response.to.have.status(200);
});

Step 4: Save your request.

2. Exporting the Postman Collection

• After saving your request and tests, click on the “…” next to the collection name.

• Select “Export” and choose the collection format (e.g., v2.1).

• Save the JSON file to your project directory.

  

3. Setting Up Newman in Your Project

Step 1: In your project directory, run the following commands to install Newman and the HTML reporter:

npm install newman
npm install newman-reporter-htmlextra --legacy-peer-deps

Step 2: Create a JavaScript file (e.g., runnewmanci.js) to configure and run Newman:

 const newman = require('newman');

 newman.run({
 collection: require('./bonus_postman_collection.json'),
 debug: true,
 reporters: ['htmlextra','cli'],
 reporter: {
     'htmlextra': {
         export: './report.html', // export html-extra report
     },
 },
 }, function (err) {
     if (err) { throw err; }
     console.log('collection run complete!');
 });

4. Integrating Newman into CI/CD

5. How Pipelines Work

1. Push Code: Developers push code to the repository.
2. Run Pipeline: GitLab automatically detects the .gitlab-ci.yml file and runs the pipeline.
3. Jobs Execution: Jobs are executed in stages, with jobs in the same stage running in parallel and stages running sequentially.
4. Review: Developers review the pipeline’s result. If all jobs are successful, the changes can be merged into the main branch.

Artifacts

Artifacts are the files created when a job runs in a pipeline. These can be used to store binary files, test results, or any other piece of information that can be retrieved at a later stage.

• In your CI/CD configuration file, set up a job to run the Newman test script. Below is an example configuration for GitLab CI/CD:

image: node:18

stages:
  - test

before_script:
  - npm install newman
  - npm install newman-reporter-htmlextra --legacy-peer-deps

test_newman:
  stage: test
  script:
    - node runnewmanci.js
  artifacts:
    paths:
      - report.html

after the sync in vs code it should be in the repo

when the gitlabrun is executed you should see results

When the job is executed then you should see an artifact

hey the test went wrong lets analyse the issue:

newman dashboard will define how many testcases went wrong

In postman we corrected the testcases functional and also the collection variable which was not set correctly and thus gave a 0 result 0 * 0

corrected the variables

Now the job succeeds

the bonus calc command line output is corrected

and also the html report

I hoped it helped you how easy it is actually to put a postman script via newman in CI happy testing!

gitlab project can be found here https://gitlab.com/learnautomatedtesting/postmanbonuscalculator. The api key in this coolection is throthled it to max 1000 request a day. This is the api key:2yCN3fpzSl2rptL24RLlaEF1XrIxAbz1WPkoYOQh at the moment (x-api) and can only be used for 400 requests a day

Conclusion

By integrating Newman with your CI/CD pipeline, you can automate the testing of your Postman collections and ensure that your APIs meet the defined requirements before deployment. This enhances the reliability and stability of your applications, making Newman a valuable tool in your development workflow.