Getting Started with the Unified Agent

Overview

This page describes how to get started with the Unified Agent.

Prerequisites

  1. Ensure you have one of the following Java versions on the computer on which you want to run the Unified Agent:

    • Java JDK 8

    • Java JRE 8

    • Java JDK 11

    • Java JDK 17

  2. Depending on your project type, make sure that the relevant package manager is installed:

Project Type

Package Manager

Project Type

Package Manager

C#

  • NuGet + .Net

  • Paket

Elixir, Erlang

MIX

Go

  • Dep

  • Godep

  • Vndr

  • Gogradle

  • Govendor

  • Gopm

  • Glide

  • Vgo

  • Modules

  • Bazel

Haskell

Cabal

Java

  • Maven

  • Gradle

  • ANT

  • Bazel

JavaScript

  • Yarn - required only if the project is not built

  • NPM - required only if the project is not built (or the detection was set to use NPM)

  • Bower

Objective-C, Swift

CocoaPods - required only if the project is not built  

OCaml

Opam

PHP

Composer - required only if the project is not built

Python

  • PIP

  • Poetry

  • Pipenv

  • Conda

R

Packrat - if used

Ruby

Bundler

Rust

Cargo - required only if the project is not built

Scala

SBT

Unified Agent Usage Overview

Step #

Step Name

Step #

Step Name

1

Download the latest version of the Unified Agent and verify its integrity.

2

Set up the Unified Agent.

3

Do one of the following:

(See execution examples on this page)

4

View the results in your Mend organizational portal.

Downloading the Unified Agent

The Unified Agent latest version can be downloaded from Amazon S3 or GitHub.

Latest Unified Agent Version

File

Features

Release Date

Latest Unified Agent Version

File

Features

Release Date

22.6.1

wss-unified-agent.jar

Release Notes 22.6.1

12-06-2022

Previous Unified Agent Versions

NOTE: Unified Agent versions will be available and supported for a year after their release.  

Version

File

Features

Release Date

MD5

Comments

Version

File

Features

Release Date

MD5

Comments

22.5.2

wss-unified-agent-22.5.2.jar

Release Notes 22.5.2

12-06-2022

N/A

N/A

22.5.1

wss-unified-agent-22.5.1.jar

Release Notes 22.5.1

29-05-2022

N/A

N/A

22.4.2

wss-unified-agent-22.4.2.jar

Release Notes 22.4.2

18-05-2022

N/A

N/A

22.4.1.1

wss-unified-agent-22.4.1.1.jar

Release Notes 22.4.1.1

02-05-2022

N/A

N/A

22.4.1

wss-unified-agent-22.4.1.jar

Release Notes 22.4.1

01-05-2022

N/A

N/A

22.3.3

wss-unified-agent-22.3.3.jar

Release Notes 22.3.3

17-04-2022

N/A

N/A

22.3.2.1

wss-unified-agent-22.3.2.1.jar

Release Notes 22.3.2.1

06-04-2022

N/A

N/A

22.3.2

wss-unified-agent-22.3.2.jar

Release Notes 22.3.2

03-04-2022

N/A

N/A

22.3.1

wss-unified-agent-22.3.1.jar

Release Notes 22.3.1

20-03-2022

N/A

N/A

22.2.2.2

wss-unified-agent-22.2.2.2.jar

Release Notes 22.2.2.2

20-03-2022

N/A

N/A

22.2.2.1

wss-unified-agent-22.2.2.1.jar

Release Notes 22.2.2.1

09-03-2022

N/A

N/A

22.2.2

wss-unified-agent-22.2.2.jar

Release Notes 22.2.2

06-03-2022

N/A

N/A

22.2.1

wss-unified-agent-22.2.1.jar

Release Notes 22.2.1

20-02-2022

N/A

N/A

22.1.2

wss-unified-agent-22.1.2.jar

Release Notes 22.1.2

06-02-2022

N/A

N/A

22.1.1.1

wss-unified-agent-22.1.1.1.jar

Release Notes 22.1.1.1

27-01-2022

N/A

N/A

22.1.1

wss-unified-agent-22.1.1.jar

Release Notes 22.1.1

23-01-2022

N/A

N/A

21.12.2

wss-unified-agent-21.12.2.jar

Release Notes 21.12.2

09-01-2022

N/A

N/A

21.12.1

wss-unified-agent-21.12.1.jar

Release Notes 21.12.1

26-12-2021

9F459173DD2F4A1B0BCA16E3387248D6

N/A

21.11.2.1

wss-unified-agent-21.11.2.1.jar

Release Notes 21.11.2.1

16-12-2021

C938CB75252B312961D16FBA3FC0F443

N/A

21.11.2

wss-unified-agent-21.11.2.jar

Release Notes 21.11.2

12-12-2021

1EEDF78497CD5801D152B19CC7E65C69

N/A

21.11.1

wss-unified-agent-21.11.1.jar

Release Notes 21.11.1

28-11-2021

BA19E92BC8B508AA93CC7652EC878B1B

N/A

21.10.2

wss-unified-agent-21.10.2.jar

Release Notes 21.10.2

14-11-2021

90A989C0D60A70FCC9A62AF2D54F5E9B

N/A

21.10.1

wss-unified-agent-21.10.1.jar

Release Notes 21.10.1

31-10-2021

74BB03F271E93CAB9FD3710FC6B786DB

N/A

21.9.1.1

wss-unified-agent-21.9.1.1.jar

Release Notes 21.9.1.1

25-10-2021

706694E349EA14CB04C4621B70D99A93

N/A

21.9.1

wss-unified-agent-21.9.1.jar

Release Notes 21.9.1

17-10-2021

510303D7588212C666F16B042A05BC3F

N/A

21.8.1.1

wss-unified-agent-21.8.1.1.jar

Release Notes 21.8.1.1

31-08-2021

239DAD256F1E8BFAF7361FDEECE3E2CE

N/A

21.8.1

wss-unified-agent-21.8.1.jar

Release Notes 21.8.1

29-08-2021

5E066881180AB1C1C24748145F784B32

N/A

21.7.2

wss-unified-agent-21.7.2.jar

Release Notes 21.7.2

15-08-2021

90BE9617B380EF507C5D5ABE0191FCFA

N/A

21.7.1

wss-unified-agent-21.7.1.jar

Release Notes 21.7.1

01-08-2021

B18E7F9CACB80993151F4518F852710B

N/A

21.6.3

wss-unified-agent-21.6.3.jar

Release Notes 21.6.3

18-07-2021

B897BADFEBA66A39963717899E327F86

N/A

21.6.2.2

wss-unified-agent-21.6.2.2.jar

Release Notes 21.6.2.2

06-07-2021

BAC44FB66BE88130ECA094A37B81F527

N/A

21.6.2

wss-unified-agent-21.6.2.jar

Release Notes 21.6.2

04-07-2021

5E7FE501C0B1BEF76F64EE683B917012

N/A

21.6.1

wss-unified-agent-21.6.1.jar

Release Notes 21.6.1

20-06-2021

F2EB843816A572904954052756EB66E7

N/A

Setting Up the Unified Agent

There are several methods for configuring the Unified Agent.

Environment Variables (Recommended)

All the parameters available in the configuration file can be passed to the Unified Agent using environment variables. For more information, refer here.

Configuration File

  • A configuration file can be passed to the Unified Agent in the command line using the -c argument. If no file is specified, the Unified Agent will look for a configuration file named wss-unified-agent.config in the current working directory.  Refer here for more information.
    It is recommended to create a blank configuration file and only add parameters that you want to change, in order to make use of the default configuration settings. As a reference, please refer here.

Command-line Parameters

The Unified Agent supports command-line options and parameters. For more information refer here.

The configuration is applied in the following order of precedence:

  1. Command-line parameters

  2. Environment variables

  3. Configuration file

  4. Default values

For the full configuration parameters reference, refer to the Unified Agent Configuration Parameters page.

Setting the Minimum Required Configuration Parameters

Set the following configuration parameters, in any of the available methods, for the Unified Agent's execution:

Parameter Name

Environment Variable Name

Configuration File Parameter Name

Command Line Parameter Name 

Description

Parameter Name

Environment Variable Name

Configuration File Parameter Name

Command Line Parameter Name 

Description

API Key

WS_APIKEY

apiKey

-apiKey

The identifier of the organization. This can be found on the Integrate page of the Mend User Interface under the Organization section. Requires admin level access to see this page.

Mend URL

WS_WSS_URL

wss.url

-wss.url

The Server URL with /agent added. This can also be found on the Integrate page of the Mend User Interface under the Organization section. Requires admin level access to see this page.

For example: https://saas.whitesourcesoftware.com/agent

User Key

WS_USERKEY

userKey

-userKey

Required. See the following link for how to generate a user key.

Product Name

WS_PRODUCTNAME

productName

-product

The name of the product created after running a scan.

Project Name

WS_PROJECTNAME

projectName

-project

The name of the project created after running a scan

Scanning Best Practices

General Tips

  • Require a userKey by enabling enforce user level access in order to see which team members are scanning. 
    NOTE: The userKey is also required for API calls and reporting parameters such as generateScanReport.

  • Optimal detection is achieved when scanning after a successful build where dependency files used to create the application are available.
    NOTE: This will allow the Unified Agent to detect libraries with all three of its detection methods, as described below.

Detection Methods

Dependency Resolution

During the detection, manifest files (such as, requirements.txt in python) are used to pinpoint a specific version of the package used.  

Binary and Source File Matching Overview

The Mend Unified Agent also detects binaries and source files (such as, .py files in Python or a .jar file in Java) and matches them against the Mend Index.

  • Mend matches binary and source files to the repository (such as, GitHub, SourceForge) from which they most likely originated. 

  • The Mend knowledge base includes ~340M files and ~45M open source projects.

  • The file matching method is required when there are no known packages that can be resolved by utilizing the dependency resolution process. 

  • For each matched source file, the likely origin of that source is determined using a proprietary algorithm: SmartMatch
    For details, see Source Files Matching Algorithm: SmartMatch

    • It is recommended to enable SmartMatch for any existing organization.

    • SmartMatch is enabled by default for any newly created organization.

  • Supported File Formats lists all currently supported file formats for hash matching.

  • Binary matches occur only for the exact hash of each file.

  • This feature can be disabled by setting fileSystemScan=false as the default value is true.

Running the Unified Agent

To run the Unified Agent from the command line, execute the following commands in a shell script task as part of your build pipeline or in the directory where your codebase is located: 

cd <your codebase directory>

Linux/macOS:

export WS_APIKEY=<your-api-key>
export WS_USERKEY=<your-user-key>
export WS_PRODUCTNAME=<your-product-name>
export WS_PROJECTNAME=<your-project-name>
export WS_WSS_URL=https://saas.whitesourcesoftware.com/agent
java -jar wss-unified-agent.jar

Windows:

set WS_APIKEY=<your-api-key>
set WS_USERKEY=<your-user-key>
set WS_PRODUCTNAME=<your-product-name>
set WS_PROJECTNAME=<your-project-name>
set WS_WSS_URL=https://saas.whitesourcesoftware.com/agent
java -jar wss-unified-agent.jar

NOTES:

  • Specify the -d parameter to scan another directory besides the current working directory.

  • Full or relative paths can be used, however paths with spaces must be double-quoted ("").

Viewing and Understanding the Scan Steps and Summary

The Unified Agent command-line interface enables you to view the steps that ran as part of a scan and understand how long each step took.

Start/End Indication

A start/end indication is displayed for each scan step. For example:

1 2 3 4 5 6 7 8 9 10 11 12 ------------------------------------------------------------------------ -------------------- Start: Pre-Step & Resolve Dependencies ------------ ------------------------------------------------------------------------ [INFO] [2019-03-07 13:58:02,775 +0200] - Trying to resolve MAVEN dependencies [INFO] [2019-03-07 13:58:02,776 +0200] - topFolder = C:\Users\Me\Desktop\UAtests\GenerateScanReport\generateScanReport\Data [INFO] [2019-03-07 13:58:07,105 +0200] - Start parsing pom files [INFO] [2019-03-07 13:58:07,112 +0200] - End parsing pom files , found : search-engine,search-engine-client,search-engine-server [INFO] [2019-03-07 13:58:07,191 +0200] - Trying to resolve HTML dependencies [INFO] [2019-03-07 13:58:09,113 +0200] - ------------------------------------------------------------------------ -------------------- End: Pre-Step & Resolve Dependencies -------------- ------------------------------------------------------------------------

Summary Table

A summary at the end of scan with all the relevant information on each step is also displayed. It Includes the following columns:

  • Step: The relevant step of the scan

  • Completion Status: Either 'COMPLETED' or 'FAILED'

  • Elapsed: The time that step took. Note that the sub-steps are not included in the total elapsed running time (e.g., Maven, HTML).

  • Comments: When available, more information on the step.

For example: 

1 2 3 4 5 6 7 8 9 10 11 12 13 Step Completion Status Elapsed Comments ====================================================================================================================================================== Fetch Configuration COMPLETED 00:00:00.078 -------- Scan Files Matching 'Includes' Pattern COMPLETED 00:00:00.014 1 source/binary files Pre-Step & Resolve Dependencies COMPLETED 00:00:06.378 7 total dependencies (7 unique) MAVEN COMPLETED 00:00:04.416 5 total dependencies (5 unique) HTML COMPLETED 00:00:01.922 2 total dependencies (2 unique) Update Inventory COMPLETED 00:00:01.551 2 updated projects ====================================================================================================================================================== Elapsed running time: 00:00:08.021 ====================================================================================================================================================== Process finished with exit code SUCCESS (0)

Execution Examples

The following are several syntax examples for various use cases of the Unified Agent execution.

Executing the Unified Agent with Inline environment variables:

export WS_APIKEY=<your-api-key>
export WS_USERKEY=<your-user-key>
WS_PRODUCTNAME=<your-product-name> WS_PROJECTNAME=<your-project-name> java -jar ./wss-unified-agent.jar

Executing the Unified Agent with the config file:

java -jar ./wss-unified-agent.jar -c /path/to/config/file -d /directory/to/scan

Executing the Unified Agent on multiple folders or files:

export WS_APIKEY=<your-api-key>
export WS_USERKEY=<your-user-key>
export WS_PRODUCTNAME=<your-product-name>
export WS_PROJECTNAME=<your-project-name>
java -jar ./wss-unified-agent.jar -d /directory/to/scan,/directory/to/scan2,/file/to/scan

Executing the Unified Agent with a policy check to return an error code in order to break a CI/CD pipeline:

export WS_APIKEY=<your-api-key>
export WS_USERKEY=<your-user-key>
export WS_PRODUCTNAME=<your-product-name>
export WS_PROJECTNAME=<your-project-name>
export WS_CHECKPOLICIES=true
export WS_FORCECHECKALLDEPENDENCIES=true
export WS_FORCEUPDATE=true
export WS_FORCEUPDATE_FAILBUILDONPOLICYVIOLATION=true
java -jar ./wss-unified-agent.jar

Executing the Unified Agent with a proxy:

export WS_APIKEY=<your-api-key>
export WS_USERKEY=<your-user-key>
export WS_PRODUCTNAME=<your-product-name>
export WS_PROJECTNAME=<your-project-name>
export WS_PROXY_HOST=<your-proxy-host-name>
export WS_PROXY_PORT=<your-proxy-port-number>
export WS_PROXY_USER=<your-proxy-username>
export WS_PROXY_PASS=<your-proxy-password>
java -jar ./wss-unified-agent.jar 

Additional examples for CI/CD pipelines and executing Mend Prioritize can be found at https://github.com/whitesource-ft/ws-examples.