Overview 

Test runs specify a set of test cases to execute and include the results of running the test cases, indicating whether actual results differ from expected results. When you create a test run, you select one or more test cases as the basis for the test run. You then schedule the test run to execute, which captures and stores values (results) for all test cases in the test run. Test then compares the returned values against the baseline values for each test case, and stores and reports the findings.

For example, a test run for use with the Salesforce CPQ Calculate Quote API specifies a number of test cases to execute to ensure that the latest Salesforce update did not introduce regressions in your CPQ pricing calculations. When you execute the test run, Test calls the Salesforce CPQ Calculate Quote API for each test case, stores the returned values in the test run, compares the results for each test against the test case’s baseline values, and stores pass/fail test run results in the test run.

A test run consists of:

• A list of test cases to execute.
• The results of executing the test cases.
• Pass/fail baseline comparison values.

The Test Runs Page 

To access the page, in the left sidebar of the Test tab, click Test Runs.

The Test Run Results tab displays pertinent information for each executed test run:

• Test Run Number – Specifies an auto-generated tracking number for the test run. This field is system-generated and read only.
• Status – Indicates the progress of the test run. This field is system-generated and read only. These values are possible:
  • Not Started – Indicates there have never been any attempts to execute the test run.
  • In Progress – Indicates the test run is currently executing. To monitor progress, navigate to the Test Run Results page.
  • Complete – Indicates the test run has finished executing. To view the results, navigate to the Test Run Results page.
• Test Cases – Specifies the number of test cases associated with the test run. Hover over the numbers to see the names of the test cases.
• Run Date – Specifies when the test run execution took place.
• Details – Specifies the total number of tests run and number of those tests that passed.
• Run By – Specifies the user who created the test template

The page also contains the Scheduled Test Runs tab.

The Scheduled Test Runs tab displays pertinent information for each scheduled test run:

• Scheduled Job Key – Specifies an auto-generated tracking number for the test schedule. This field is system-generated and read only.
• Description – Displays the optional description you previously specified.
• Test Cases – Specifies the number of test cases associated with the test run. Hover over the numbers to see the names of the test cases.
• Next Scheduled Run (UTC) – Specifies the next time test run will execute.
• Recurrence (UTC) – Specifies the schedule of future runs.

The Test Run Results Page 

To access the page, from the Test Runs page, on the Test Run Results tab, click any Test Run Number.

The first table displays pertinent test run result information for each test case Test executed:

• View Details – Displays the Test Result Details table for the given test case.
• Test Case Name – Specifies the name of the test case executed in the test run.
• Test Case ID – Specifies the ID of the test case executed in the test run.
• Function – Specifies the type of functional test executed.

• Result – Visually indicates the result of the executed test case with one of these icons:
  •  All tests passed – The test case execution was 100% successful.
  •  One or more tests failed – The test case execution was less than 100% successful.
  •  Error – There was an error that prevented the test case from fully executing.

The second table displays pertinent test result details for each tested field in the test case you selected from the first table and is sortable on any column:

• Link to Record – Provides quick, direct access to the Salesforce object record containing the field. Use the link to analyze fields for tests that didn’t pass.

• Test Result Number – Specifies a system-generated name for the results record.
• Tested Object– Specifies the object containing the tested field.
• Tested Field – Specifies the field tested.
• Details – Specifies the result details. These values are possible:
  • Expected: <n>, Actual: <m> – Specifies the value returned for the field and the expected corresponding baseline value.
  • Tested Field not found – Signifies that no value was returned by the API.
• Result – Visually indicates the result of the tested field with one of these icons:
  •  Test passed – The tested field matches.
  •  Test failed – The tested field does not match.
  •  Error – There was an error that prevented the test from fully executing.