Newman Run Action
The Newman Run action in Continua is a wrapper around the newman.exe command line. If you're having trouble using the Newman action, please refer to the Command Line Reference.
Newman Run is a command-line Collection Runner for Postman. It can be use to run API tests and report on the outcomes.
Newman
Name
A friendly name for this action (will be displayed in the actions workflow area).
Collection File
The path and filename of the Postman collection file. This is a JSON file which can be exported from Postman.
Working Directory
Where to run the Newman command line. Newman will use this working directory while reading files with relative paths. Defaults to workspace directory. [--working-dir]
Folder
The name of a single folder to run within the Postman collection. [--folder]
Iteration Count
Number of iterations to run. [--iteration-count]
Install Newman CLI on agent using NPM
Tick this to automatically install the Newman CLI on the agent using NPM. A new NPM tab will also be displayed with options for the NPM command line.
Using
When not installing using NPM, the "Using" drop down is populated with any property collector whose namespace matches the pattern defined by the Newman action. The pattern for this action is ^Newman\..*
The default property collector will search the environment PATH for "newman.exe". If you create a property collector for this action, make sure you select the Path Finder PlugIn type and give it a name that will match the pattern above in blue.
For more in-depth explanations on property collectors see Property Collectors.
Alternatively, you can select the Custom option from the Using drop down list and specify a path in the resulting input field that will be displayed. Please read Why it's a good idea to use a property collector before using this option.
NPM
Install Newman CLI To
The folder NPM should install the Newman CLI to.
Using NPM
The Using drop down is populated by any property collector properties whose namespace matches the pattern defined by the NPM action. The pattern for this action is ^NPM\..* . The default property collector searches the environment path for "NPM.exe".
If you create a property collector for this action, make sure you select the Path Finder PlugIn type and give it a name that will match the pattern above in blue.
For more in-depth explanations on property collectors see Property Collectors.
Alternatively, you can select the Custom option from the Using drop down list and specify a path in the resulting input field that will be displayed. Please read Why it's a good idea to use a property collector before using this option.
Files
Environment File
The path to a JSON file containing Postman environment variables. [--environment]
Globals File
The path to a JSON file containing Postman global variables. [–globals]
Iteration Data File
The path to a JSON or CSV file containing data to use for Postman iterations. [--iteration-data]
Cookie Jar File
The path to a JSON file containing a JSON cookie jar in tough-cookie format. [--cookie-jar]
Variables
Postman Global Variables
Global variables to pass to the Postman collection. One key=value pair per line. [--global-var]
Postman Environment Variables
Environment variables to pass to the Postman collection. One key=value pair per line. [--global-var]
Settings
Stop the runner when a test case fails.
Tick to abort the collection run when a test case fails. [--bail]
Prevent reading of files outside of the working directory
Untick this to allow reading files external to the working directory. [--no-insecure-file-read]
Request Delay
Delay (in milliseconds) between requests. [--delay-request]
Timeout
Time (in milliseconds) to wait for collection run to complete execution. Leave blank (or 0) for no timeout. [--timeout]
Request Timeout
Time (in milliseconds) to wait for each request. Leave blank (or 0) for no timeout. [--timeout-request]
Script Timeout
Time (in milliseconds) to wait for scripts to complete execution. Leave blank (or 0) for no timeout. [--timeout-script]
Suppress exit code
Tick to continue running tests even after a failure, but exit with code equal to 0. [--suppress-exit-code]
Suppress all output
Tick to run without any output to the log file. [--suppress-exit-code]
Show detailed information of collection run and each request sent
Tick for verbose output. [–verbose]
SSL
Turn off strict SSL
Tick to disable SSL certificate validation. [--insecure]
SSL Client Certificate List
The path to a JSON file containing client SSL certificate configurations. [--ssl-client-cert-list]
SSL Client Certificate
The path to a PEM client SSL certificate file. [--ssl-client-cert]
SSL Client Key
The path to a JSON file containing client SSL certificate private key. [--ssl-client-key]
SSL Client Passphase
An SSL client certificate passphrase (for protected key). [--ssl-client-passphrase]
SSL Extra CA Certificate
The path to an additionally trusted CA certificate. [--ssl-extra-ca-certs]
Export
Collection Export File
The path to a file to export the final the Postman collection JSON to before completing the run. [--export-collection]
Environment Export File
The path to a file to export the final the Postman environment JSON to before completing the run. [--export-environment]
Globals Export File
The path to a file to export the final the Postman global JSON to before completing the run. [--export-globals]
Cookie Jar Export File
The path to a file to export the final the Postman cookie jar JSON to before completing the run. [--export-cookie-jar]
Reporters
Reporters
Optionally select one or more reporters for outputting the test results to. The JUnit XML reporter is required to display results on the Unit Tests tabs of the Build page. The output from HTML reporter can optionally be automatically registered as a report. Note that the CLI reporter will be added automatically so that the you can view results in the build log. [--reporter]
Use HTML Extra reporter
Shown when the HTML reporter option is selected. Tick to use the HTML Extra reporter to produce a richer HTML report. Note that the HTML Extra reporter is not built-in to newman and must be installed.
Use JUnit Full reporter
Shown when the JUnit XML reporter option is selected. Tick to use the JUnit Full reporter to produce a richer Junit report. Note that the JUnit Full reporter is not built-in to newman and must be installed.
Custom Reporter
The name of a custom reporter to use.
Additional Reporter Arguments
Additional arguments for reporters. One argument value pair per line.
Install Newman HTML Extra Reporter on agent using NPM
Tick to automatically install the Newman HTML Extra Reporter using the same NPM settings used to install Newman
Install Newman JUnit Full Reporter on agent using NPM
Tick to automatically install the Newman JUnit Full Reporter using the same NPM settings used to install Newman
Register all tests as unit tests
Shown only when the JUnit reporter is selected. Tick to register test results with Continua so that that are displayed on the Unit Test tab of the Build page.
Automatically register HTML report with the server
Shown only when the HTML reporter is selected. Tick to automatically create the necessary workspace rules and report registration to add the HTML report to the build Reports tab. Note that the HTML report export file needs to be in the workspace to register the report with the server.
HTML Reporter Export Path
Shown only when the HTML reporter is selected. The path to export the HTML report to. Defaults to a unique file in the 'newman-reports' subfolder of working folder. [--reporter-html-export | --reporter-htmlextra-export]
HTML Reporter Template Path
Shown only when the HTML reporter is selected. The path to a custom Handlebars template used to render the HTML report. [--reporter-html-template | --reporter-htmlextra-template]
JUnit Reporter Export Path
Shown only when the JUnit reporter is selected. The path to export the JUnit report to. [--reporter-junit-export | --reporter-junitfull-export]
JSON Reporter Export Path
Shown only when the JSON reporter is selected. The path to export the JSON report to. [--reporter-json-export].
Report
Name
The name of the report - shown on the report selection tab. This defaults to a name derived from the name of the action.
Description
A description for the report - shown as the tooltip title on the report selection tab.
Run Order
The order in which the reports are displayed. Reports with higher run order numbers appear to the right of those with lower numbers. If run order numbers are equal then reports are ordered by report creation time.
Options
Fail action if any tests fail
Tick this to cause the build to fail if any tests fail.
Fail action if any tests error
Tick this to cause the build to fail if an error occurred while running any test.
Fail action if any tests are inconclusive
Tick this to cause the build to fail if any tests are inconclusive.
Timeout (in seconds)
How long to wait for the action to finish running before timing out. Leaving this blank (or zero) will default to 86400 seconds (24 hours).
Treat failure as warning
Tick to continue build on failure marking the action with a warning status.
Ignore warnings
If this is ticked, any warnings logged will not mark the action with a warning status.
Environment
Environment Variables
Multiple environment variables can be defined - one per line. These are set before the command line is run.
Log environment variables
If this is ticked, environment variable values are written to the build log.
Generate system environment variables
Tick this checkbox to set up a list of new environment variables prefixed with 'ContinuaCI.' for all current system expression objects and variables.
Mask sensitive variable values in system environment variables
This checkbox is visible only if the 'Generate system environment variables' checkbox is ticked.
If this is ticked, the values of any variables marked as sensitive will be masked with **** when setting system environment variables. Clear this to expose the values.