2021.06.02

What You Should Already Know

If you’re following along from your own computer, by the end of Part 1, you should have a working Postman request that POSTs a test case execution result to a test cycle in Jira’s Zephyr Scale app. If you haven’t caught up from Part 1 yet, start there: http://ms-wit.com/how-i-run-tests-in-postman-and-report-the-results-to-jira-with-zephyr-scale-part-1/

What You Should Have

In addition to everything we used in Part 1, you should also have an API you can work with. If you don’t have one of your own, you can use Postman’s Echo API. I work with some private APIs in my day job, but for this, I too will use the Echo API.

Let’s Continue

As its name suggests, the Echo API merely returns, echos back to you, whatever you send it. This makes it ideal for writing tests that will pass and can be forced to fail. After all, a test you can’t force into failure may not be a very good test. The Echo API also comes with some nifty Utilities, like the one I’m using here.

My first request is a GET request to the Current UTC Time

GET 'https://postman-echo.com/time/now'

And for a response, I get something like:

Wed, 02 Jun 2021 01:30:17 GMT

Probably, one of the first Postman tests you learned to write is a test for the http response code. If the request above works properly, it should return an http response code of 200 OK. Our test for such a GET request would look like so:

// Expect an HTTP response code of 200
pm.test("Status Test", function () {
    pm.response.to.have.status(200);
});

So far so good. Or if you’ve been writing Postman tests for a while, so far so what. We’re not telling Zephyr Scale anything at all yet. There are at least a couple ways to combine this test result from the Echo GET request with the Zephyr Scale POST request from Part 1, including chaining the requests together, using postman.setNextRequest(), and passing the result from the Echo GET request as a variable into the next (Zephyr) POST request.

Alas, I often have a sequence of requests to make in a particular order and I don’t want the POSTs to Zephyr getting in the middle of my sequence. This is where it gets a little complicated and where pm.sendRequest comes to the rescue. We can use this pm.sendRequest function in a test of my Echo GET request and it would POST the result to Zephyr all in one go.

Recall in Part 1, I showed how I could preview the cURL of my POST to Zephyr.

The cURL I need is on the right.

Using that cURL, I’ve added 15 lines to my tests tab and now I have this:

// Expect an HTTP response code of 200
pm.test("Status Test", function () {
    pm.response.to.have.status(200);
});
zephyrAuth = pm.globals.get("ms-wit_zephyr");
// Report to Zephyr
pm.test("Report to Zephyr.", function () {
  pm.sendRequest({
    url: 'https://api.adaptavist.io/tm4j/v2/testexecutions',
    method: 'POST',
    header: {
      'Content-Type': 'application/json',
      'Authorization': zephyrAuth
    },
    body: '{"projectKey":"JIRA","testCaseKey":"JIRA-T1","testCycleKey":"JIRA-R1","statusName":"pass","testScriptResults":[{"statusName":"pass","actualResult":"I can pass an execution IN a Postman test"}]}'
  }, function (err, res) {
    pm.globals.set("token", res.json().access_token);
  });
});

Let’s unpack that. The first 4 lines are my original test, expecting a response of 200 from the GET to Echo’s Current UTC Time endpoint.

Line 5 is how I get the value of my own Zephyr authorization token from where it is stored as a global variable. I’ll use it in line 13.

Lines 6 and 7 are just my standard opening for the new “test” and line 8 is the beginning of the pm.sendRequest function.

Lines 9-14 are common elements needed for the sendRequest function; a url, a method, a header, and a body.

Line 15 is a string of the whole payload of data that makes up what gets reported to Zephyr. Look closely and you’ll see the Project Key, the Cycle Key, the Case Key, the pass/fail status, and the Result Notes.

Here’s what the Request’s Test tab looks like now, with the Zephyr report inside a pm.sendRequest function.

Hopefully, you’ve also noticed two important things here.

  1. For now, I’ve faked the body of that sendRequest including the pass/fail Status and the Result Notes in this example.
  2. If you’re as lazy and clever as I am, you see a lot of opportunities to use variables, particularly if you don’t want to write the Result Notes by hand for each request which may vary by environment, passed parameters, or pre-request scripts and all.

So, with the sendRequest function like the above as our goal, I need to build it with actual results instead of magic strings. To do that I need to capture those values and pass them into that long payload string; the body of the sendRequest. So I wrote two more “tests” that are executed before the actual results are POSTed. The variables that I’ll need for these two new “tests” are:

passedTests = 0;
zephyrResult = "Fail";
zephyrResultNotes = "";
testCycleKey = pm.environment.get("ZEPHYR_TEST_CYCLE");
testEnv = pm.environment.get("URL");
testRunner = "Name of My Computer";
zephyrAuth = pm.globals.get("ms-wit_zephyr");
testCaseKey = "JIRA-T1";

That may be more than you were expecting, and the variables you need for your own tests will probably be different. Let’s unpack all those variables so you can decide what to keep, what to discard, and what to add for your own work.

passedTests = 0; I set this to zero at the beginning and increment its value for each assertion-based test that needs to pass in order for the whole test case to pass.

zephyrResult = “Fail”; I set this to fail at the beginning so that something positive has to happen in order to flip the value to “Pass”. If something positive fails to happen in echo assertion-based test, the zephyrResult will remain “Fail”

zephyrResultNotes = “”; I set this to an empty string at the beginning so that the report to Zephyr won’t fail because of a null value in the event something goes very wrong with the request.

testCycleKey = pm.environment.get(“ZEPHYR_TEST_CYCLE”); I change this environment variable often, based on the kind of testing that’s underway. e.g the CI/CD pipeline has its own dedicated cycle to report to, as does each Regression session.

testEnv = pm.environment.get(“URL”); I use this in the Result Notes as an indicator of which environment was tested. It’s not strictly relevant to the Echo API, but I wanted to demonstrate how useful it can be to build the Result Notes with variables from the Request being tested.

testRunner = “Name of My Computer”; I work with a team of testers in my day job, so I use this in the Result Notes as an indicator of what computer was used to make the Request.

zephyrAuth = pm.globals.get(“ms-wit_zephyr”); This was originally on line 5. I just moved it to the top with the rest of the variables. It’s the value of my Zephyr Authorization token.

testCaseKey = “JIRA-T1”; This is unique to each Request. Later, in Part 3, I’ll copy all but this last variable into an environment variable so that I can have access to them all with one line of code.

Now that I have all this data as variables, I can write one of the new “tests” to build the body of the POST request that’s being sent to Zephyr. Actually, the body is itself going to become a string variable.

passedTests = 0;
zephyrResult = "Fail";
zephyrResultNotes = "";
testCycleKey = pm.environment.get("ZEPHYR_TEST_CYCLE");
testEnv = pm.environment.get("URL");
testRunner = "Name of My Computer";
zephyrAuth = pm.globals.get("ms-wit_zephyr");
testCaseKey = "JIRA-T1";
// Expect an HTTP response code of 200
pm.test("Status Test", function () {
  pm.response.to.have.status(200);
  passedTests++;
});
// Determine the final results and notes for Zephyr. This API Request has 1 test that must pass.
pm.test("Determine results for Zephyr Reporting.", function () {
  if (passedTests == 1) {
    zephyrResult = "Pass";
    zephyrResultNotes = testRunner + " with Postman was looking for and found the current UTC time from " + testEnv;
  } else {
    zephyrResultNotes = testRunner + " with Postman failed to find the current UTC time from " + testEnv;
  }
  return zephyrResultNotes;
});
// Get and set some variables to prepare for Zephyr reporting
pm.test("Build Zephyr reporting request.", function () {
  let bodyBuilderJson = {
    'projectKey': 'JIRA',
    'testCaseKey': testCaseKey,
    'testCycleKey': testCycleKey,
    'statusName': zephyrResult,
    'testScriptResults': [
      {
        'statusName': zephyrResult,
        'actualResult': zephyrResultNotes
      }]
  };
  let bodyBuilder = JSON.stringify(bodyBuilderJson);
  pm.environment.set("ZEPHYR_BODY_BUILDER", bodyBuilder);
});
// Actual Zephyr reporting.
pm.test("Report to Zephyr.", function () {
  var bodyBuilder = pm.environment.get("ZEPHYR_BODY_BUILDER");
  pm.sendRequest({
    url: 'https://api.adaptavist.io/tm4j/v2/testexecutions',
    method: 'POST',
    header: {
      'Content-Type': 'application/json',
      'Authorization': zephyrAuth
    },
    body: bodyBuilder
  }, function (err, res) {
    pm.globals.set("token", res.json().access_token);
  });
});

First of all, don’t miss the new line inside the Status Test, line 12 above. If the Status Test passes, i.e. the GET request to Echo returns a ‘200 OK’ response code, the passedTest variable will be incremented from 0 to 1.

The next “test,” beginning at line 16, isn’t so much a test as it is a way to determine the pass/fail status of the tests as a whole, based on the value of passedTests, and to programmatically build the string that’s used in the Result Notes being POSTed to Zephyr.

Because there is currently only one assertion based test that must pass, the if statement in line 16 is expecting the passedTests variable to be 1, that’s the positive thing that needs to happen for the zephyrResult to flip from “Fail” to “Pass”. No matter what happens with the status, the resultNotes string is built accordingly using the relevant values from the actual GET request, like the computer making the GET request and the endpoint of the request.

Finally, given we now have real results stored in variables, we can programmatically build the body of what will eventually become the pm.sendRequest() that actually POSTs to Zephyr. I refer to this “test” as the Body Builder. It begins at line 25. Using all our new variables, it creates the json needed for the sendRequest function, converts that json into a string, and stores that string for the final “test,” the POST to Zephyr. Look, no more magic strings or faked data. This improved POST to Zephyr begins on line 41 and produced an execution result like the one pictured below.

Passing result in Zephyr with actual data, no magic strings.

Stay tuned for Part 3 of this tutorial, where I will take what I’ve done here and expand on it with another test or two, force a failure, and make as much as possible reusable for other requests.