Scanning with Multi-Module Analysis

Overview

The Multi-Module Analysis feature enables you to analyze large Project structures comprising multiple Project modules. The analysis results obtained for each module are displayed in the application within an independent Project corresponding to the analyzed module; the relevant Projects are located under a single Product corresponding to the multi‑module structure.

This feature enables you to analyze the sub modules, referenced from a main module, by referencing parameters for the main module without being required to explicitly specify parameters for each of the 'sub' modules. It provides the user with a command that inspects the configuration for a specified multi-module and creates a setup file featuring the appPath and Project name references for the submodules reflecting the inspection. The resulting setup file is referenced within a WhiteSource Unified Agent command.

Prerequisites

Multi-Module Analysis is supported for Java and JavaScript Projects . Also supported are Projects of a wider scope that feature any mixture of Java, Kotlin, Scala & JavaScript

  • The following Unified Agent parameters must be set to True:

    • gradle.aggregateModules

    • maven.aggregateModules

    • enableImpactAnalysis

  • The following Unified Agent parameters must be set to False:

    • gradle.runPreStep

    • maven.runPreStep

    • npm.runPreStep

Running the Unified Agent with the '-analyzeMultiModule' Parameter 

-analyzeMultiModule inspects the configuration for a specified multi-module and creates a setup file featuring application path (AppPath) references that reflect the inspection.
A Unified Agent command with the following parameters is required to create the setup file for the multi-module definitions:

Parameter

Description

Parameter

Description

-analyzeMultiModule 

Instructs the command to inspect the structure of a specified multi-module (rather than run Effective Usage Analysis) and save the project name for each sub-module in a setup file

-d

The argument text value specifies the path for the main module's pom (e.g., c:\test1)

-overrideExistingSetup

When set to True, enables users to override an existing multi-module setup file when running the first step of the multi-module WhiteSource Prioritize (default is False).

The following is the command that inspects the structure of a specified multi-module and generate a file ('setupFile.txt' or any other file name that the user defines) for multi-module analysis:

1 java -jar wss-unified-agent.jar -d <Path For The Main Module's Project Directory> -analyzeMultiModule setupfileName 

A sample command:

1 java -jar wss-unified-agent.jar -d C:\Users\JSmith\euaMultiModuleAnalyzer\ -analyzeMultiModule c:\test\samplefolder\sampleSetupFile.txt

The following message is displayed after the setup file was created successfully:

[INFO] <time_stamp>] - The multi-module analysis setup file was created successfully.

If any of the values were left blank, the following warning message is also displayed: 

[WARN][<time_stamp>] - Analysis found multiple candidates for one or more appPath settings that are listed in the multi-module analysis setup file. Please review the setup file and set the appropriate appPath parameters.

In such cases, the 'appPath' value(s) must be entered manually. See the next section for more information on the AppPath parameter.

Setup File Structure & Contents

The following is the structure of the setup file contents:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 DependencyManagerFilePath=<Path to the Dependency Manager file> ProjectFolderPath1=<Path to the first file project folder> AppPath1=<Path to the first file project application (e.g., Jar file> defaultName1=<Default name of the first file project> altName1=<Alternative name of the first file project> ProjectFolderPath2=<Path to the second file project folder> AppPath2=<Path to the second file project application (e.g., Jar file> defaultName1=<Default name of the second file project> altName1= <Alternative name of the second file project> .... ProjectFolderPathN=<Path to the 'N' file project folder> AppPathN=<Path to the 'N' file project application (e.g., jar file> defaultNameN=<Default name of the 'N' file project> altNameN= <Alternative name of the 'N' file project>

The file includes the following contents:

  • DependencyManagerFilePath: The path to the dependency manager file folder. The first line of the generated file includes this parameter and its value. There is only one instance of this parameter in the file and it recommended not to manually modify its value. 

  • ProjectFolderPath<number of instance>: The folder name associated with a Project (e.g., c:\a\project\project1). Its value is automatically obtained based on analysis. 

    • The setup file can include multiple instances of the ProjectFolderPath parameter. The parameter must be specified in an independent line and have a corresponding 'appPath' parameter specified in the subsequent line. It can be removed only if the corresponding 'appPath' is removed as well.

  • AppPath: The filename associated with a Project (e.g., c:\a\project\project1\tester.jar). The value is automatically obtained based on analysis.

    • There may be cases where the AppPath value is left blank, and in such cases, the value must be manually entered. 

To prevent cases where inadvertently running multi-module WhiteSource Prioritize with duplicate Project name entries result in ignoring processes, the system includes these two parameters that, when manually entered, eliminate the scenario where multiple Projects have the same name. By default. both of these parameters have the value of the Project name.

  • defaultName: Default name of the file Project (editable)

  • altName: Alternative name of the file Project (editable)

The following is a sample of the setup file:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 DependencyManagerFilePath=C:/Users/JohnSmith/Documents/Apps/CloudApp/project ProjectFolderPath1=C:/Users/JohnSmith/Documents/Apps/CloudApp/project/buildSrc AppPath1=C:/Users/JohnSmith/Documents/Apps/CloudApp/project/buildSrc/build/libs/buildSrc.jar defaultName1=buildSrc altName1=buildSrc ProjectFolderPath2=C:/Users/JohnSmith/Documents/Apps/CloudApp/project/services/api AppPath2=C:/Users/JohnSmith/Documents/Apps/CloudApp/project/services/api/build/libs/api-1.0.jar defaultName2=api altName2=api ProjectFolderPath3=C:/Users/JohnSmith/Documents/Apps/CloudApp/project/services/webservice AppPath3=C:/Users/JohnSmith/Documents/Apps/CloudApp/project/services/webservice/build/libs/webservice-2.5.war defaultName3=webservice altName3=webservice ProjectFolderPath4=C:/Users/JohnSmith/Documents/Apps/CloudApp/project/api AppPath4=C:/Users/JohnSmith/Documents/Apps/CloudApp/project/api/build/libs/api-1.0.jar defaultName4=api altName4=api_1

The following is a sample of the setup file that has missing 'AppPath' values:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 DependencyManagerFilePath=C:/Users/JSmith/Desktop/euaMultiModuleAnalyzer/ksa AppPath1=C:/Users/JSmith/Desktop/euaMultiModuleAnalyzer/ksa/ksa-service-root/ksa-logistics-service/target/ksa-logistics-service-3.9.0.jar ProjectFolderPath1=C:/Users/JSmith/Desktop/euaMultiModuleAnalyzer/ksa/ksa-web-root/ksa-bd-web defaultName1=ksa altName1=ksa AppPath2= ProjectFolderPath2=C:/Users/JSmith/Desktop/euaMultiModuleAnalyzer/ksa/ksa-web-root/ksa-finance-web defaultName2=ksa-finance-web altName2=ksa-finance-web AppPath3=C:/Users/JSmith/Desktop/euaMultiModuleAnalyzer/ksa/ksa-web-root/ksa-finance-web/target/ksa-finance-web-3.9.0.jar ProjectFolderPath3=C:/Users/JSmith/Desktop/euaMultiModuleAnalyzer/ksa/ksa-dao-root/ksa-bd-dao AppPath3=C:/Users/JSmith/Desktop/euaMultiModuleAnalyzer/ksa/ksa-dao-root/ksa-bd-dao/target/ksa-bd-dao-3.9.0.jar defaultName3=ksa-bd-dao altName3=ksa-bd-dao ProjectFolderPath4=C:/Users/JSmith/Desktop/euaMultiModuleAnalyzer/ksa/ksa-web-root/ksa-web AppPath4= defaultName4=ksa-web altName4=ksa-web

In the above sample, 'AppPath2' and 'AppPath4' must be entered manually.

Downloading & Running the xModuleAnalyzer 

Overview

The xModuleAnalyzer for WhiteSource Prioritize enables you to specify the .jar path of the Unified Agent, configuration file of the Unified Agent, and the relevant setup file that was created in the previous step. 

The output analyzes each submodule listed in the multi-module setup file in a threshold mode display (also the default display status when no value is provided) or a dynamic mode display. 

Downloading the xModuleAnalyzer

Refer here for the latest version of the xModuleAnalyzer:

Version

File

Release Date

Comments

Version

File

Release Date

Comments

21.8.1

xModuleAnalyzer.jar

29-August-2021

  • Align xModuleAnalyzer with latest UA (21.8.1)

Version

File

Release Date

Comments

Version

File

Release Date

Comments

21.8.1

xModuleAnalyzer-21.8.1.jar

29-August-2021

  • Align xModuleAnalyzer with latest UA (21.8.1)

21.7.2

xModuleAnalyzer-21.7.2.jar

15-August-2021

  • Align xModuleAnalyzer with latest UA (21.7.2)

21.7.1

xModuleAnalyzer-21.7.1.jar

01-August-2021

  • Align xModuleAnalyzer with latest UA (21.7.1)

21.6.3

xModuleAnalyzer-21.6.3.jar

18-July-2021

  • Align xModuleAnalyzer with latest UA (21.6.3

21.6.2

xModuleAnalyzer-21.6.2.jar

04-July-2021

  • Align xModuleAnalyzer with latest UA (21.6.2

21.6.1

xModuleAnalyzer-21.6.1.jar

20-June-2021

  • Align xModuleAnalyzer with latest UA (21.6.1

21.4.1

xModuleAnalyzer-21.4.1.jar

25-April-2021

  • Align xModuleAnalyzer with latest UA (21.4.1

21.3.2

xModuleAnalyzer-21.3.2.jar

11-April-2021

  • Align xModuleAnalyzer with latest UA (21.3.2)

20.8.1

xModuleAnalyzer-20.8.1.jar

30-August-2020

  • Aggregate Modules mode supported (using -aggregateModules field).

20.4.1

xModuleAnalyzer-20.4.1.jar

26-April-2020

  • Align xModuleAnalyzer with latest UA (20.4.1)

20.3.2

xModuleAnalyzer-20.3.2.jar

12-April-2020

  • Setting memory preference for whole xModuleAnalyzer with -EuaMaxTotalMemAlloc flag 

  • Setting preference for xModuleAnalyzer to ignore EUA  return code with -ignoreEuaNotices

  • Printing to log directory, to file with format ( xModuleAnalyzerLog_ddmmyyyy_hhmmss.txt) which EUA codes returned without  EUA000 according to -ignoreEuaNotices as well as failed processes (not necessarily due to EUA failures).

  • To use this version, please use UA versions >= 20.3.2

20.3.1


xModuleAnalyzer-20.3.1.jar

29-Mar-2020

  • Minor fix

20.2.2

xModuleAnalyzer-20.2.2.jar

19-Mar-2020

  • Avoid collision between processes 

  • Add G1 garbage collector

20.2.1

xModuleAnalyzer-20.2.1.jar

01-Mar-2020

  • User can control how to run the UA's processes with -runInParallel

19.9.2.2

xModuleAnalyzer-19.9.2.2.jar

27-OCT-2019

  • Minor fix

19.5.2

xModuleAnalyzer-19.5.2.jar

02-JUN-2019

  • Support JavaScript project

19.2.2

xModuleAnalyzer-19.2.2.jar

10-Mar-2019

  • New CLI params  -mode (Analysis level for EUA) and -productName

19.1.1

xModuleAnalyzer-19.1.1.jar

27-Jan-2019

  • Supports automatic/default and manual exclusion of files

18.10.3

xModuleAnalyzer-18.10.3.jar

18-Nov-2018



Running the xModuleAnalyzer 

Command Line Parameters

Parameter 

Type 

Description 

Required?

Default

Parameter 

Type 

Description 

Required?

Default

-xModulePath

String

Setup file pathname

Yes



-fsaJarPath

String

Unified Agent .jar pathname

Yes



-c

String

Configuration file for Unified Agent.

Yes



-aggregateModules

Boolean

If True - aggregates the results of all scanned modules to a single unified Project.

No

False

-statusDisplay

String

Screen display mode. See Threshold Mode Display and Dynamic Mode Display for details.

No

threshold

-logPath

String

Path for storing logs created by the analysis.

No

<DependencyManagerFilePath>\Whitesource-Logs

-productName

String

Product name. Overrides the -product value of Unified Agent.

No



-runInParallel

Boolean

Whether to run the processes of the Unified Agent in parallel (faster) or serial (less prone to errors) mode.

No

true

-EuaMaxTotalMemAlloc

String

EuaMaxTotalMemAlloc [custom_val] | DEFAULT | MAXIMIZE

Governs the allowed memory allocation for WhiteSource Prioritize.

  • Allowable values:

    • [custom_val] : A value entered by the user (in GB)

    • DEFAULT (default): The value provided by WhiteSource

    • MAXIMIZE : Allocate all available memory for the new process (WhiteSource Prioritize will use the available memory as needed for all modules - not per module)

No

The value provided by WhiteSource

-ignoreEuaNotices

String

Specifies how WhiteSource Prioritizes codes will be handled.

This parameter governs the ability to ignore specified Prioritizes codes for analyzed modules.

  • Allowable values:

    • all: xModuleAnalyzer will return 0 regardless of any codes returned by modules being analyzed

    • informational (default): xModuleAnalyzer will return 0 only if the codes returned by all modules being analyzed are either [EUA000] or of class Informational. Otherwise, the code returned will be -100.

    • none: xModuleAnalyzer will return 0 only if the codes returned by all modules being analyzed are [EUA000]. Otherwise the code returned will be -100.

No

informational

Command Structure

1 java -jar xModuleAnalyzer-<x.x.x>.jar -xModulePath setupfileName -fsaJarPath unifiedAgentJarPathName -c configFileForUnifiedAgent -statusDisplay <dynamic|threshold>

<x.x.x> represents the latest version number of the specified Jar file.

Returned Codes for xModuleAnalyzer

xModuleAnalyzer can return the following codes:

  • (Success) 0: All multi-module analyses completed successfully (all modules returned [EUA000], or all codes were ignored based on a supported setting).

  • (Error) -100: Some modules were not analyzed successfully (one or more modules returned a non-[EUA000] code and codes were not ignored based on a supported setting).

  • (Failure) -200: It was not possible to successfully meet pre-conditions for xModuleAnalyzer.

Also, if one or more modules in a multi-module setting failed to run or complete (not due to an EUA-related error), a corresponding informational message will be returned by xModuleAnalyzer. Note that the returned xModuleAnalyzer code will not be influenced by the conditions leading to the message (e.g., a return code of 0 (success) will not change if one or more modules did not complete processing).

The message will be formatted as follows (it will be captured in the designated xModuleAnalyzer log - see Log Files):

1 2 3 4 5 6 7 8 9 The processing of the following modules did not complete: Module module_identifier Module module_identifier … Module module_identifier

Threshold Mode Display (Default)

In this mode, the screen display is only refreshed whenever analysis progress for each Project crosses a specified threshold  (25%, 50%, 75%, and 100% thresholds), and only if a specified interval of one minute has elapsed. The following is a sample command for a threshold mode display command that stores the logs in the path 'C:\logs\':

1 java -jar xModuleAnalyzer.jar -xModulePath C:\Users\JSmith\Desktop\euaMultiModuleAnalyzer\ksa\setupFile.txt -fsaJarPath C:\Users\JSmith\git\unified-agent\target\wss-unified-agent.jar -c C:\Users\JSmith\Desktop\NPM-Plugin\wss-unified-agent.config -statusDisplay threshold

Dynamic Mode Display

In this mode, the screen display is refreshed every 10 seconds to show the analysis progress for each Project. The following is a sample command for a dynamic mode display command:

1 java -jar xModuleAnalyzer.jar -xModulePath C:\Users\JSmith\Desktop\euaMultiModuleAnalyzer\ksa\setupFile.txt -fsaJarPath C:\Users\JSmith\git\unified-agent\target\wss-unified-agent.jar -c C:\Users\JSmith\Desktop\NPM-Plugin\wss-unified-agent.config -statusDisplay dynamic

Each analyzed Project has one of the following statuses for each phase of analysis: '

  • Pending

  • In-Progress

  • Success

  • Failure

Log Files

After the process completes, a log file is generated for each module. Log files are located in the following path: <DependencyManagerFilePath>\Whitesource-Logs.

This path can be modified by using the optional -logPath parameter. For example, in the following command, the log path is C:\logs\.

1 java -jar xModuleAnalyzer.jar -xModulePath C:\Users\JSmith\Desktop\euaMultiModuleAnalyzer\ksa\setupFile.txt -fsaJarPath C:\Users\JSmith\git\unified-agent\target\wss-unified-agent.jar -c C:\Users\JSmith\Desktop\NPM-Plugin\wss-unified-agent.config -statusDisplay threshold -logPath C:\logs\


When the analysis finishes with errors, an indication to examine the related log file is displayed. All completion statuses that include the '( * )' characters indicate that errors were detected.

Regardless of the above log files, the following log files will be created:

  1. xModuleAnalyzerLog_ddmmyyyy_hhmm.log - This file will include all the console log of the multi-module analysis.

  2. projectDetails{ddmmyyyy_hhmmss}.json - This file will include the output of the Unified Agent scan (similar to the fsa.json) in order to clarify how the Unified Agent divided the modules.