...

...

...

...

Supported File Extensions

For a complete list of supported file extensions (binary files, source files, and archive files) that the Unified Agent supports, refer here.

Selecting a Plugin for Integration

...

Analyzing Archives Files

If you have Java/Ruby/Python archive files and you are willing to open them and extract descriptive information, then you can use this feature by providing values for the following parameters:

• archiveExtractionDepth 
• archiveIncludes 
• archiveExcludes

The drill-down hierarchy is limited to a maximum of 10 and can be modified in the configuration file.
By default, the drill-down hierarchy level is zero - no drill down.
Supported archive types are as follows:

• aar
• car
• ear
• egg
• gem
• hpi
• jar
• nupkg
• rar
• rpm
• sca
• sda
• tar
• tar.bz2
• tar.gz
• tar.xz
• tgz
• war
• whl
• xz
• zip

NOTES:

...

Supported File Extensions

For a complete list of supported file extensions (binary files, source files, and archive files) that the Unified Agent supports, refer here.

Selecting a Plugin for Integration

Scanning Archives Files

If you have Java/Ruby/Python archive files and you are willing to open them and extract descriptive information, then you can use this feature by providing values for the following parameters:

• archiveExtractionDepth 

• archiveIncludes 

• archiveExcludes

The drill-down hierarchy is limited to a maximum of 10 and can be modified in the configuration file.
By default, the drill-down hierarchy level is zero - no drill down.
Supported archive types are as follows:

• .aar

• .car

• .ear

• .egg

• .gem

• .hpi

• .jar

• .nupkg

• .rar

• .rpm

• .sca

• .sda

• .tar

• .tar.bz2

• .tar.gz

• .tar.xz

• .tgz

• .war

• .whl

• .xz

• .zip

Providing a Project Name Only in a Unified Agent Scan

Overview and Default Behavior

If a scanned project uses a configuration file that contains only a defined projectName (where projectToken, productName and productToken are left empty), then the Unified Agent adds the results to the first project it finds in the organization that contains the same project name.
If a multi-module project is scanned with the same configuration as described above, and a project with one of the module names already exists, then the Unified Agent adds the module data to this existing project.

Example: Only 'projectName' is defined and 'projectToken’, ‘productName’ and ‘productToken’ are left empty

projectName=NewProject
projectToken=
 
productName=
productToken=

Changing the Default Behavior

In order to avoid overriding an existing project with the same name, from the Advanced Settings section of the Integrate page, select the checkbox Add project to default product when only project name is provided.

When this checkbox is selected, the following rules apply to all future scans:

• When only projectName is provided in the Unified Agent configuration (as demonstrated in the above example), the Unified Agent adds the project to a default product named 'My Product'. This rule is implemented only when the default product 'My Product' does not already contain a project with the same name.

• If the default product 'My Product' already contains a project with the name specified in the 'projectName' parameter, then the Unified Agent returns an error to the user with the message "Project <PROJECT_NAME>' already exists in 'My Product'. A 'productToken' or 'productName' value should also be specified".

• If a multi-module project is scanned, and a project with one of the module names already exists under your product, an error is returned and none of the projects are created as part of the scan.

Configuration Recommended Mode

...

The generated configuration will contain the list of recommended parameters based on the environment, and also the mandatory parameters: APIKey, ProductName, ProjectName/ProjectToken and Wss.URL. 

To use the Configuration Recommendation mode, do as follows:

1. In the folder that you want to scan, in the command line, enter java -jar unified_agent.jar -detect. 

2. In the folder that was scanned, access the generated file, wss-generated-file.config. 

3. Add the mandatory parameters to the file. 

4. Run the Unified Agent scan: java -jar unified_agent.jar -c wss-generated-file.config.

The includes parameter is supported by this feature, enabling WhiteSource to automatically identify the environment that the user wants to scan and create the configuration file automatically.

...

These exit codes are also available after the process ends, in the environment's last command's return value variable

(Linux: $?,  PowerShell:  $LASTEXITCODE,  Batch: %ERRORLEVEL%)

Exit Codes in Bash

The exit codes WhiteSource returns in the Bash command language should be treated as 'x' modulo 256: 

• Exit code 0 is equivalent to code 0 (0 mod 256 = 0)

• Exit code -1 is equivalent to code 255 (-1 mod 256 = 255)

• Exit code -2 is equivalent to code 254 (-2 mod 256 = 254)

• Exit code -3 is equivalent to code 253 (-3 mod 256 = 253)

• Exit code -4 is equivalent to code 252 (-4 mod 256 = 252)

• Exit code -5 is equivalent to code 251 (-5 mod 256 = 251)

• Exit code -6 is equivalent to code 250 (-6 mod 256 = 250) 

Using Proxy Settings 

For customers using a proxy, the relevant proxy details are defined here.

Use the CLI parameter ‘-proxy’ to allow for an easy method to set proxy settings. The following command allows you to download a remote configuration file using proxy settings:

NOTE: The following protocols are supported: 'file://', 'ftp://', 'http://', 'https://'

Best Practices for Offline Mode (Optional) 

Overview

It 's is possible to save the output of the scan into a file instead of sending it directly to WhiteSource by HTTPS. This approach is useful in case there is no connectivity (or limited connectivity) while scanning.

By changing the configuration file to offline mode, any execution of the Unified Agent will store the current configuration and metadata in a JSON .txt file named update-request.txt, located in the newly-created 'whitesource' directory. It is located under the current working directory ($pwd or %cd%). This file can later be manually uploaded to WhiteSource from the Admin Console or via the command line.

...

Update the wss-unified-agent.config file:

• Change the offline property to true.

• For very large projects, it's recommended that you add the property offline.zip=true to reduce the size of the created file. This setting enables zipping the content of the output file.

• To prettify the output file to JSON format, add the offline.prettyJson=true property.

Uploading an Offline Request File 

Upload via command line with configuration file (supported from version 1.8.9): 

Before you begin this procedure, ensure these parameters are properly configured:

• apiKey

• wss.url

• projectName

Execute the Unified Agent with the -requestFiles flag specifying the path to the update-request.txt file you created in the previous step. In order to send more than one file, separate file names with a comma. 

java -jar <path to wss-unified-agent-*.jar> -c <path of config file> -requestFiles <file-path1>[,<file-path2>]

Upload via command line without configuration file (supported from version 1.8.9): 

• Execute the Unified Agent with the -requestFiles flag specifying the path to the update-request.txt file you created in the previous step. In order to send more than one file, separate file names with a comma.

java -jar <path to wss-unified-agent-*.jar> -wss.url <WhiteSourceInstance/agent> -apiKey <apiKey> -productToken <productToken> -project <newProjectName> -requestFiles <pathToUpdate-request.txt> -noConfig true

...

It is recommended to perform this procedure per release. Do as follows:

1. Download

   JarSigner

   JarSigner (there are multiple sources from where the utility can be downloaded).

2. From the command line, enter the following command to run JarSigner and view the list of security certificates in the .jar file:

After running, ensure that the WhiteSource information appears in the list of security certificates.

Scanning Remote Repositories 

Overview

For customers using SCM (Git, SVN, Mercurial), refer here for the relevant settings.

If the source code is not located on your machine, it is possible to provide connection information to your remote repository using Git, SVN or Mercurial. Note the following:

• Only descriptive information is sent to our servers. All of the processing is done locally on your machine.

• Dimensions CM is also supported by using either Git via Git Connector or Subversion via CM Bridge. 

• You cannot scan both a local repository and a remote repository simultaneously. If you are intending to scan a local

...

• repository and not a remote repository, then use the

...

• -d

...

• parameter instead of the SCM parameters that are described below. 

Prerequisites

The scm client must be installed on your machine in order to successfully connect to your repository:

• Git

• SVN

• Mercurial

WhiteSource also supports on-premises installations of repositories. 

Usage in Different Repository Types

• For Git repositories, provide the repository URL and the branch or tag name. If no branch or tag is defined, it will default to the 'master' branch.

• For SVN repositories, provide the URL for the specific trunk, branch or tag. If no branch or tag is defined, it will default to the 'trunk' branch.

• For Mercurial repositories, provide the repository URL and the branch or tag name. If no is branch or tag is defined, it will default to the 'master' branch.

NOTE: If local files are to be scanned, then remove or comment out the scm parameters.

Multiple Remote Repositories

Multiple remote repositories can be scanned by creating a JSON file, similar to the following example:

{

...

"scmRepositories":[

...

    { "url":"path/to/remotreRepository1", "branch":"ChooseBranch", "tag":"ChooseTag" },

...

    { "url":"path/to/remotreRepository2", "branch":"master", "tag":"0.2" }

...

    ]

...

}

Remote Repository (SSH)

SSH is currently supported only for Git repositories.

Set the scm.ppk property to your private key file path and set the scm.pass property to the passphrase of the private key. If there are none, then leave empty.

Proxy for Remote Repository

If you have a proxy for your remote repositories, add these java parameters:

-Dhttps.proxyHost=PROXY_HOST -Dhttps.proxyPort=PROXY_PROT

The 'failErrorLevel' Parameter 

Default Behavior 

When the  'failErrorLevel ' parameter is set to 'DEFAULT', the behavior of the exit codes stay stays the same.

Modifying the Default Behavior 

When the  'failErrorLevel ' parameter is set to 'ALL', the Unified Agent returns an error code for all errors in the scan, rather than 'SUCCESS'. This can be useful for customers migrating from the plugins (NPM, MAVEN etc.) to the Unified Agent.

The parameter sets additional scenarios to 'error ' instead of 'success'.The following cases are considered as 'Failure':

• resolver.runPrepStep failed (at least one)

• Error in collecting/running dependencies (at least one)

• When ‘productName’ and ‘productToken’ are missing, and no ‘projectToken’ is defined in the configuration file.

Examples:

• npm.resolveDependencies=true, but resolve dependencies failed

• nuget.runPreStep=true, but the pre-step failed

• mvn dependency:tree failed or partial data was retrieved
  

...

A summary report in JSON format can be automatically generated locally at the end of each scan, using the 'generateScanReport' configuration parameter when running the Unified Agent.
This report includes information on vulnerabilities, policy violations, top fixes and inventory details.

The default filename format of the JSON report is '<project_name>-<yyyy-mm-dd>T<HHmmss>+<UTC offset>-scan_report.json'. 
For example: 'Demo App-2019-06-04T181226+0300-scan_report.json'

The following configuration parameters are available to control timeouts and file name format for the report:
scanReportTimeoutMinutes: Time-out (in minutes) for the process of generating the scan report. If the timeout interval has passed then the report will not be generated, but the scan will continue.
scanReportFilenameFormat: Controls filename format of a generated scan report.

NOTES:

• Only Organization and Product Administrators can generate this report. The 'userKey' configuration parameter is mandatory for this option.

• In order to generate this report, the configuration parameter 'updateInventory' must be set to true.

• If 'checkPolicies' is set to true then 'forceUpdate' should also be set to true.

• If 'checkPolicies' is set to false then no policy related data will be generated in the report.

The following is an example of a scan report with custom attributes available on the project:

{
  "projectVitals": {
    "productName": "NuspecTest",
    "name": "NuspecTest_4",
    "token": "d38f6222562b403jiiojioc4e4cd19c7d54d6206b8bb5",
    "creationDate": "2019-02-27 13:52:46",
    "lastUpdatedDate": "2019-03-20 07:32:48"
  },
  "libraries": [
    {
      "keyUuid": "86d115e2-99ab-4jd-8092-f510b14fe949",
      "keyId": 35111144,
      "name": "microsoft.aspnetcore.server.kestrel.transport.abstractions.2.1.3.nupkg",
      "groupId": "",
      "artifactId": "microsoft.aspnetcore.server.kestrel.transport.abstractions.2.1.3.nupkg",
      "version": "",
      "sha1": "b1ef47c06e9e884doijooeafafafs0dfc83f3",
      "type": "UNKNOWN_ARTIFACT",
      "licenses": [],
      "vulnerabilities": [],
      "matchType": "SHA1",
      "customAttributeValues": [
        {
          "contextName": "eRez-wss",
          "context": "Organizational",
          "contextId": "18945",
          "attr1": "rfv",
          "contextToken": "88cd28476aec411bjojojojojoi0283068a4632a4eae439bf70e691"
        }
      ],
      "policyDetails": {
        "policyName": "",
        "status": "no policy applied"
      }
    },
    {
      "keyUuid": "018fijo9-aa6c-4118-bl[[l-b0b918b64311",
      "keyId": 35109719,
      "name": "microsoft.aspnetcore.2.1.4.nupkg",
      "groupId": "",
      "artifactId": "microsoft.aspnetcore.2.1.4.nupkg",
      "version": "",
      "sha1": "0706d598c463aede3fjojoioiioj5810q3tq3t4371a935e9df92a5",
      "type": "UNKNOWN_ARTIFACT",
      "licenses": [],
      "vulnerabilities": [],
      "matchType": "SHA1",
      "customAttributeValues": [
        {
          "contextName": "eRez-wss",
          "context": "Organizational",
          "contextId": "18945",
          "attr1": "edc",
          "contextToken": "88cd28476aec4d68e411b02808a4632a4eae439bf70e691"
        }
      ],
      "policyDetails": {
        "policyName": "",
        "status": "no policy applied"
      }
    },
    {
      "keyUuid": "b3c82c1a-8864-4842-bqwer-193df8683f04",
      "keyId": 35111034,
      "name": "system.memory.4.5.0.nupkg",
      "groupId": "System.Memory",
      "artifactId": "system.memory.4.5.0.nupkg",
      "version": "4.5.0",
      "sha1": "5835a676479b3e6a32167606f6",
      "type": "NUGET_PACKAGE_MODULE",
      "languages": "Nuget",
      "references": {
        "url": "https://api.nuget.org/packages/system.memory.4.5.0.nupkg",
        "homePage": "https://dot.net/",
        "genericPackageIndex": "https://api.nuget.org/packages/System.Memory/4.5.0"
      },
      "licenses": [
        {
          "name": "MIT",
          "url": "http://www.opensource.org/licenses/MIT",
          "profileInfo": {
            "copyrightRiskScore": "THREE",
            "patentRiskScore": "ONE",
            "copyleft": "NO",
            "royaltyFree": "YES"
          },
          "referenceType": "NuGet package (details available in nuget gallery)",
          "reference": "https://github.com/dotnet/corefx/blob/master/LICENSE.TXT"
        }
      ],
      "vulnerabilities": [],
      "matchType": "SHA1",
      "customAttributeValues": [],
      "policyDetails": {
        "policyName": "mit",
        "status": "reject"
      }
    },
    {
      "keyUuid": "5d63e066-b54f-43c3-b254",
      "keyId": 22769402,
      "name": "microsoft.aspnet.razor.3.2.3.nupkg",
      "groupId": "Microsoft.AspNet.Razor",
      "artifactId": "microsoft.aspnet.razor.3.2.3.nupkg",
      "version": "3.2.3",
      "sha1": "2cfd6d2ea3eb89362fa21a9b47",
      "type": "NUGET_PACKAGE_MODULE",
      "languages": "Nuget",
      "references": {
        "url": "https://api.nuget.org/packages/microsoft.aspnet.razor.3.2.3.nupkg",
        "genericPackageIndex": "https://api.nuget.org/packages/Microsoft.AspNet.Razor/3.2.3"
      },
      "licenses": [
        {
          "name": "Microsoft .NET Library",
          "url": "http://microsoft.com/web/webpi/eula/aspnetcomponent_rtw_enu.htm",
          "riskLevel": "unknown",
          "referenceType": "NuGet package (details available in nuget gallery)",
          "reference": "http://www.microsoft.com/web/webpi/eula/net_library_eula_ENU.htm"
        }
      ],
      "vulnerabilities": [],
      "matchType": "FILENAME",
      "customAttributeValues": [
        {
          "contextName": "eRez-wss",
          "context": "Organizational",
          "contextId": "18945",
          "attr1": "1qa",
          "contextToken": "88cd28476aec4d68e429c181kjv3068a4632a4eae439bf70e691"
        },
        {
          "contextName": "NuspecTest_4",
          "context": "Project",
          "contextId": "479597",
          "attr2": "2ws",
          "contextToken": "d38f6222562b46c2a8popb7f7c4e4cd19c7d54d6206b8bb5"
        },
        {
          "contextName": "NuspecTest",
          "context": "Product",
          "contextId": "78835",
          "attr3": "3ed",
          "contextToken": "822ec9add1c84ebmomo20711cd41cmomo7a80de57c035a43"
        }
      ],
      "policyDetails": {
        "policyName": "policy1",
        "status": "reject"
      }
    },
    {
      "keyUuid": "e178fa80-3196-49e8-8753emomoec8f",
      "keyId": 22769401,
      "name": "jquery.validation.1.11.1.nupkg",
      "groupId": "jQuery.Validation",
      "artifactId": "jquery.validation.1.11.1.nupkg",
      "version": "1.11.1",
      "sha1": "d6a4e4a2ccb2eaa203c0a99fb5168585d",
      "type": "NUGET_PACKAGE_MODULE",
      "languages": "Nuget",
      "references": {
        "url": "https://api.nuget.org/packages/jquery.validation.1.11.1.nupkg",
        "homePage": "http://bassistance.de/jquery-plugins/jquery-plugin-validation/",
        "genericPackageIndex": "https://api.nuget.org/packages/jQuery.Validation/1.11.1"
      },
      "licenses": [
        {
          "name": "MIT",
          "url": "http://www.opensource.org/licenses/MIT",
          "profileInfo": {
            "copyrightRiskScore": "THREE",
            "patentRiskScore": "ONE",
            "copyleft": "NO",
            "royaltyFree": "YES"
          },
          "referenceType": "NuGet package (details available in nuget gallery)"
        }
      ],
      "vulnerabilities": [],
      "outdated": true,
      "matchType": "FILENAME",
      "outdatedModel": {
        "outdatedLibraryDate": "2013-03-25",
        "newestVersion": "1.17.0",
        "newestLibraryDate": "2018-01-25",
        "versionsInBetween": 6
      },
      "customAttributeValues": [],
      "policyDetails": {
        "policyName": "mit",
        "status": "reject"
      }
    },
    {
      "keyUuid": "9abmo4d7-6112-47cd-98f26momo9264",
      "keyId": 22756758,
      "name": "entityframework.6.1.3.nupkg",
      "groupId": "EntityFramework",
      "artifactId": "entityframework.6.1.3.nupkg",
      "version": "6.1.3",
      "sha1": "b6e3e77f9b84b21b42cbdomo975ca81a",
      "type": "NUGET_PACKAGE_MODULE",
      "languages": "Nuget",
      "references": {
        "url": "https://api.nuget.org/packages/entityframework.6.1.3.nupkg",
        "genericPackageIndex": "https://api.nuget.org/packages/EntityFramework/6.1.3"
      },
      "licenses": [
        {
          "name": "Microsoft .NET Library",
          "url": "http://microsoft.com/web/webpi/eula/aspnetcomponent_rtw_enu.htm",
          "riskLevel": "unknown",
          "referenceType": "Project home page",
          "reference": "https://www.microsoft.com"
        }
      ],
      "vulnerabilities": [],
      "outdated": true,
      "matchType": "FILENAME",
      "outdatedModel": {
        "outdatedLibraryDate": "2015-03-10",
        "newestVersion": "6.2.0",
        "newestLibraryDate": "2017-10-26",
        "versionsInBetween": 0
      },
      "customAttributeValues": [],
      "policyDetails": {
        "policyName": "policy1",
        "status": "reject"
      }
    }
  ],
  "policyStatistics": {
    "totalApproved": 0,
    "totalRejected": 4,
    "totalReassigned": 0,
    "totalConditions": 0,
    "totalIssues": 0,
    "totalLibrariesAffected": 2
  },
  "vulnerabilityStatistics": {
    "low": 0,
    "medium": 0,
    "high": 0,
    "newVulnerabilities": 0,
    "oldVulnerabilities": 0,
    "ancientVulnerabilities": 0,
    "totalOutdated": 0,
    "totalVulnerableOutdated": 0,
    "totalLowVulnerabilities": 0,
    "totalMediumVulnerabilities": 0,
    "totalHighVulnerabilities": 0,
    "totalVulnerable": 0
  }
}

Troubleshooting

• If there is an issue with the Java heap size, depending on the machine resources, try to install a 64-bit Java Runtime and use the '–d64' along with the '–Xmx' and '–Xms' switches to increase the Java heap size.

  ”

Limitations

• The minimum file size for scanning is 512 bytes. The maximum file size for scanning is 2 GB

  . All other files will be skipped and not scanned by the Unified Agent

  .

• The Unified Agent supports UTF-8 locales. If other locales are in use, the Unified Agent generates an error when confronting special characters.

• Requests with more than one million dependencies will fail.

• The Maven dependencies detection is based on the Apache Maven Dependency Plugin. Maven projects that use version 3.2.0 of the Maven Dependency Plugin are not supported.