The following figure shows the user interface of the Report Analyzer.

The following actions can be performed on the start screen:

• Change to main screen

• Load data record

• Save record

• Change application settings (see App Settings)

• Change display settings (see View Settings)

• Change export settings (see Export Settings)

• Contact support

The start page offers the following information and setting options:

• Application version number

• used license key

• Link to this documentation

• Link to the releases

Save data set

With this function all loaded and pre-filtered executions including the defined groups can be stored as a single data set. Changes made like MeasurementFilter are also taken into account. If a data set is needed again and again, this can speed up the work considerably. The originals of the reports are not needed for this. The analysis data and graphs are not part of the data set, these data will be recalculated if necessary.

Load data set

A previously saved data set can be loaded either by clicking the corresponding Open button or by using the list of recent projects.

Window arrangement

The arrangement and size of all windows or panels can be adjusted by the user according to his own ideas. To do this, a panel is held at the title bar and moved. Then drag the mouse pointer with the mouse button held down to the corresponding docking symbol in the target area. The panels can be arranged on top of each other, side by side or overlapping.

The panel arrangement is saved when the program is closed. When the program is restarted, the panels are automatically arranged as last configured.

The arrangement of the panels can be reset to a standard layout using the "Reset Layout" button in the main toolbar:

Visible columns

In most tables, only some of the available columns are displayed. Additional columns can be shown or hidden user-specifically.

By right-clicking on the table header line and selecting the Show Column Chooser entry, a dialog opens in which the columns to be displayed can be selected.

This allows the active columns to be set by checkboxes and their order by drag & drop. This mechanism is basically valid for all tables available in the program

Sorting by columns

By left-clicking on the header of a column, the table is sorted in ascending or descending order by the selected column. A triangle symbol indicates the sorting direction.

Filtering the columns

For large tables or data sets, individual rows can be filtered user-specifically. This function is especially useful for preselecting and defining groups

The tables can be filtered in three different ways:

1. by values (single column)

2. by rules (single column)

3. filter editor (several columns)

Filter by values (single column)

If you move the mouse over a column header, the filter symbol appears. Left-clicking on it opens a menu with an active tab Filter Values, which lists all possible values of this column. The user can now select the desired values by clicking into the checkbox. The search function in the upper part of the window can be used to narrow down the selection. The functionality is similar to that in Microsoft Excel.

Filter by rules (single column)

If you move the mouse over a column header, the filter symbol appears. Clicking on it opens a menu in which you have to switch to the left tab Filter Rules. In this tab you can set a rule for filtering the column.

Filter Editor (multiple columns)

The Filter Editor can be used to create extensive filter expressions. It is opened by right-clicking on the table header line and selecting the Filter Editor.

With the help of the filter editor several filter operations can be created and linked by boolean operations.

Conditional formatting

Similar to the conditional formatting in Excel, cells of a table can be specially displayed according to certain rules. For example, cells can be colored or bars or symbols can be displayed. The dialog for configuring conditional formatting can be accessed by right-clicking on the header of a column and selecting the Conditional Formatting option.

Program functions in detail

The software is available in several languages. The screenshots of this user manual have been made in the English version. The names of the windows, buttons, etc. are therefore also in English in this manual and may differ from the actual representation of the respective language.

System requirements

• Windows 7, 8, 10 (32/634 bit)

• 4GB RAM minimum, 8GB recommended

• .NET 4.6.2 or higher

Installation

To install the program, run the installer and follow the instructions.

Activation

The software must be activated with a valid license to use it. For a period of up to 60 days the software can be tested in trial mode. The trial mode is identical to the licensed version in terms of functionality. Only the created exports (PDF, Word, etc) are watermarked.

For activation a valid license key is required to unlock the software. The license key must be activated once on the IRS server.

In the tab Grouping the loaded data set can be filtered and grouped according to properties of the executions. The groups formed here are the basis for further analysis.

Groups are a central topic in the Report Analyzer. They are used to compare data sets according to arbitrarily defined criteria. These can be different product types, but also different users or characteristics of the test item or the test environment. In MSA analysis, groups are given additional significance in certain cases.

The prerequisite for this is that there is also data that allows grouping. Please refer to the section on optimizing report data (adding header information)!

For all created groups, a statistical analysis is performed for each measurement. Some analysis tools allow to compare the results of the groups.

The figure below shows the layout of the Grouping Tab. This consists of three sections:

• Upper half: Overview of all available Executions, either as a flat list or in groups, depending on how they are arranged. Here you can select the executions that should be created as new group(s).

• Bottom left: Overview of all already created groups (Active Groups). These can be moved, renamed or deleted. A subsequent addition of executions is not possible.

• Bottom right: Overview of all executions of the group selected on the left. If necessary, executions can be deleted here. Results will be recalculated.

Procedure

1. The upper view is configured using grouping (drag a column header in the grouping area above the table header) or filtering as desired

2. via the Add Group(s) button all groups or executions selected via checkbox will be added as new group(s). If nothing is selected, all currently visible executions and groups will be added.

3. In the list of active groups, the group name can be adjusted (F2) and the order can be changed. In the order defined here, the analyses will later be executed, displayed and structured in the export. The order can also be changed later. The selected groups are removed via the "Trash" button after confirmation.

4. A (re)calculation of the analysis is started automatically when leaving the grouping page.

General rules & tips for groups

• Simplified group naming can be activated in the application settings. In this case, only the values without the column names are used as groups. (e.g. instead of "ProductId: xyz" only "xyz").

• Groups can be added or deleted later. The analysis and graphs will then be recalculated. If the corresponding group is no longer available, it will be removed from the graph.

• Individual executions can be selectively deleted from already created groups. When leaving the grouping page, a new analysis will be started.

• Not all executions have to be assigned to a group.

• Only executions in groups are analyzed.

• An execution can also be assigned to several groups and will then be considered in both groups during the analysis.

Attention When the program is closed, all filter settings are saved and automatically applied when the program is restarted. If you have set a filter when creating the group, when loading the next dataset it may happen that all executions over the filter are hidden. If you remove the filter, the data will be visible again.

Grouping via filter

This procedure is recommended for the following scenarios:

• No grouping necessary (corresponds to the creation of exactly one group).

• Grouping by time ranges

• Complex groupings that cannot be realized via automated grouping

Procedure

1. the executions are filtered so that only the required entries are visible Only these entries are included in the group

2. click on Add Group(s) to create a group with a default name (e.g. Group_0)

3. with F2 the group can be renamed

4. steps 1-3 can be repeated as often as you like to create multiple groups

Grouping by columns

The fastest way to structure the data set is to use automated grouping. To do this, simply drag the column header to be grouped into the grouping area above the table header.

Procedure

1. Drag and drop the desired column header into the empty area above the table ("Drag a column header here to group by that column.")

1. The program now groups the data according to the values of the column you selected. The table immediately displays all the groups that have been formed as a preview. Use the arrow symbol to view the contents of the group.

1. Click Add Group and the groups are created with a default name and added to the list under Active Groups. The group names can be renamed with F2 if necessary.

Combined grouping

Automatic groupings can also be filtered, allowing you to combine both approaches. For example, you can group by ProductID and then use filters to select only some of the automatically created groups:

Manage groups

The button Assignments allows an analysis of the distribution of executions to groups.

A table lists all executions and informs about the number of assignments and the assigned groups to an execution. This makes it easy to detect wrong or unwanted multiple assignments or even forgotten groups.

For this purpose the table offers additional predefined filters:

• All: Shows all imported executions

• Duplicates: Displays only those executions that are assigned to multiple groups

• Unrelated: Displays only those executions that are not assigned to a group

In the main view of the Grouping Tab there is also the possibility to remove unassigned executions and thus free memory. To do this, just click on Cleanup.

Save/Apply filter set

Creating groups can be very time-consuming, especially creating complex filter expressions using the Filter Editor. Therefore, it is possible to save the current grouping visible in the Executions as a filter set and to load and apply this set again later.

Apply filter set If a filter set is to be applied, it only needs to be selected from the drop-down list. The settings take effect immediately. Existing other filters are reset.

Manage Filter Sets With the button Manage Filter-Sets the filter sets can be managed:

Export: The selected filter set is exported as an XML file and can be copied to other computers, for example Import: A previously exported XML filter set can be imported Delete: The selected filter set is removed from the list

The main screen consists of only two areas:

• Tool bar

• Tabs

Toolbar

Tabs

Tabs represent certain workflow steps and guide the operator through the program:

• Import

• Grouping

The following tabs are available if import and grouping tasks are completed:

• History and Statistics

• Error Distribution

• Graph Comparison

• Similarity Analysis

• Trend Analysis

• Watchlist

A tab consists of several areas and usually contains its own menu bar. In this menu bar, tab-specific buttons are displayed which the user can use to call up special functions. For example, the display can be changed, the help can be called up or the data export can be started.

This module shows the statistical analysis for each measurement, visualizes the history of the corresponding measured values and provides a histogram. The statistical key figures are calculated for each measurement for all groups to allow the comparison of measurements within different groups. The history diagram also allows the graphical comparison between groups. The results can be easily exported in different formats.

• Select the desired measurement and group. The view is adjusted accordingly.

• To create an MSA type 1 report, press Report Wizard.

Measurement Analysis Table (Panel "Group")

This table shows the statistical analysis values (MSA type 1) for the selected group and measurement. The page supports two different views: Group First / Measurement First (Default). Depending on the selected view, the table shows either a comparison of the groups based on the measurement or a list of all measurements for the selected group.

Meaning of the columns:

Common MSA analysis parameters by company

Here is a short overview of the usual settings for Cg and Cgk in different standards and companies. Data without guarantee! In case of doubt, please ask the company for the parameters used!

History Chart

This view shows all measurement values available for the selected measurement in a dynamic diagram. The values can be arranged by timestamp, global index, index group or serial number.

Certain elements can be shown or hidden. These include markers, the diagram legend and the limit values. The area between the lower specification limit and the upper specification limit can be filled optionally. Also the limit errors can be highlighted to get a quick overview of all errors.

The diagram also supports various views, display options and X-axes:

• Single: Only graph for the selected group is shown

• All-in-one: The graphs of all available groups are superimposed. Here it is recommended to switch off the limit value display. With the checkboxes of the legend you can hide unwanted graphs

• Stacked horizontally: The graphs of all groups are displayed side by side in their own diagrams

• Stacked vertical: The graphs of all groups are displayed one below the other in their own diagrams

With the mouse wheel or by dragging an area with the mouse the shown section of the graph can be enlarged / zoomed. After zooming, the visible section can also be moved with the mouse. For "stacked" views, the zoom is applied to all diagrams simultaneously. By clicking the "Reset Zoom" button the view can be reset to 100%.

Display options

• Lines (directly connected points)

• Points

• Bars

• Spline (Interpolated Curve)

• Steps (points connected as steps)

X-Axis Options

• Index Global: X-axis is oriented according to the index position of the execution in a global comparison (sorted by execution start)

• Index Group: X-axis is oriented according to the index position of the execution within the selected group (sorted by execution start)

• Timestamp: X-axis represents the absolute time

• Serial Nr: Here the points are sorted according to the serial number of the test object

Watchlist Function Clicking on a measured value in the diagram opens a detail table that displays all available information for the corresponding test step and test execution. Within this window, an execution can be enriched with comments and added to the watchlist to mark it for further analysis. The corresponding report can also be opened.

Histogram (panel)

Representation

The histogram diagram shows the statistical distribution of measured values of the selected measurement. Value ranges are automatically grouped and the number of measured values that lie within these limits are displayed as bars. Within these bars the status is also grouped by Passed (green) and Failed (red).

The groups are formed around the mean value in the range +/-6 sigma. The number of groups can be defined in the application settings. If the number is set to 0, the system calculates a reasonable value depending on the number of measured values.

The histogram also shows the expected normal distribution around the arithmetic mean (Xg/Avg), as well as the critical limits +/-3 sigma and, if possible, the limits LSL and USL and the nominal value (Xm). The latter depend on whether the limits within the measurement series are unambiguous and valid.

Treatment of inconsistent limits

If the limits within the series differ from each other, it is usually not possible to calculate values that depend on the lower and upper specification limits (Nominal, T, Cg, Cgk, Cp, Cpk, %EV, ...). In this case, the narrowest occurring limits are used as a substitute for calculation and the values are displayed in the table in italics. The found value ranges are displayed in the histogram as bands, which mark the largest and smallest occurring limit values:

Treatment of extreme values

Values outside the +/- 7-sigma range are visualized in the histogram in two special groups as gray bars. Values contained here (independent of the bar width) then lie outside the range of Xg +/-7 sigma:

Measurement Values (Panel)

This view shows all measured values in a sortable and filterable data table. The values can be sorted by clicking on a column header. Additionally, each column header has a symbol for filtering the values. In the lower area, sum information such as Min, Max, Avg is offered for selected columns.

Report Wizard (Button)

This button starts a dialog that helps to select the contained measurements and groups for the report.

The selected composition can also be saved and reloaded. Click on Show Preview to open the preview of the report. From here, the Export button can be used to save the file in various target formats (PDF, Word, Excel, HTML, CSV).

Report Current View (Button)

Pressing this button opens the currently selected measurement including all measured values, diagram and histogram as preview without further configuration. With the Export button, the document can be saved in various formats (PDF, Word, Excel, HTML, CSV).

Export Table (button)

The Export Table button allows the selected measurement to be exported in CSV format.

The import area is displayed by selecting the Import tab and consists of three sub-areas:

• Menu bar / Toolbar

• Import sources

• Imported Executions

The Report Analyzer allows the import from different data sources. For each data source type individual plugins are available. Depending on the version, the number of available plugins can vary. Additional plugins can be used for example to import data from CSV files or databases. The imported data is displayed in the Imported Executions area.

Import Sources (Panel)

All available import plugins are listed here. Each Plugin appears as button and if necessary an additional button for specific settings. By clicking the button the Plugin dialogue is opened, which initiates the respective import process.

All test protocols imported by the plugin are displayed in the list of imported executions.

IRS XML Plugin

The IRS XML Import Plugin is a file-based import plugin that can load IRS test reports.

The plugin supports the following files:

• Uncompressed files in IRS format: *.xml, *.irp

• Compressed files in IRS format: *.gz, *.irpz

• Appended Reports (multiple executions within one test report)

In the import dialog the files to be imported are selected. A report can contain one or any number of executions. All executions contained in the report file will be imported automatically.

Procedure

1. Select the appropriate file filter (default: *.irp). The selected filter is automatically saved and will be kept until the next change.

2. Select the base path using the Browse button ... or select a path from the dropdown menu of recently selected paths. The Refresh button refreshes the file list in the left pane if necessary and lists all subfolders and files starting from the base path.

3. The files to be imported are selected via the checkboxes in front of the respective folder or file name. You can select complete folders incl. subfolders or single files

4. (optional) Configure Import Options to optimize RAM usage or add custom header information

5. click on Import to start the import process.

Import with search filter The dialog offers a search field, which can be used to search within file names. The previous selection of files is retained! When clicking on whole folders in search mode, however, only the displayed files are added or removed. This is e.g. very helpful to filter only successful (or only failed) tests already during import!

Import via Drag & Drop At the bottom of the dialog there is a "Drop Area". Drag files or folders to this area to add them automatically to the selection. Attention: Search terms or the current file filter will be ignored! If the selection is successful, the number of detected files will be updated. A list is not displayed.

Starting the import process By clicking on "Import" an attempt is made to load all selected files. During the import it is checked if the Execution ID of the report has already been loaded. In this case the file will be ignored. After the import process is finished, a summary is displayed:

After confirmation the dialog closes and the newly imported reports are listed.

Import options

As of version 1.7, the reports to be imported can already be filtered or post-processed during the import. This allows an optimized use of memory by discarding information that is not needed for evaluation, such as info texts, additional results or "Done" steps. In addition, missing header attributes can now be added later during loading, allowing for better grouping in the evaluation. It is also possible to replace or correct existing header attributes.

Please note that the report must first be fully loaded before post-processing can take place. Therefore the import options always need some additional computing time!

Date If activated, test executions will be discarded that are not within the specified time period (the reports will still be loaded first!).

Content

• Remove 'Additional Results': removes additional data added by "Additional Results" or a check mark at "Log" in TestStand. Such data is only visible in the report, but cannot be analyzed by the Report Analyzer. Removing it can cause a significant reduction in data size

• Remove "Infotext": Removes the TestStand report text. Depending on the type of test, this can also save considerable space

Status. Here the test steps can be removed by status. If "Keep first error step" is selected, the first "Failed" or "Error" step is kept in any case. In this way, for example, an optimized error distribution can be analyzed, which only contains the "Failed" step that led to the test failure.

Header Missing header data can be added here to allow subsequent grouping. Existing header values can also be corrected here.

Settings With the checkmark "Save as default" the current settings will be used for future imports

IRS Test Report Structure For better understanding, the general data structure of an IRS report is shown here, as it is then processed by the Report Analyzer. However, the Report Analyzer only uses the "Execution" level and below after the import:

• File

  • Report

    • Execution

      • Attribute (Meta Data)

      • Attribute

      • TestStep

      • TestStep

        • Attribute (Additional Results)

      • TestStep

    • Execution

      • Attribute (Meta Data)

      • Attribute

      • TestStep

      • ...

  • ...

Execution Properties An execution contains both defined property fields (Properties) and individually usable attributes. The predefined properties are supported in all tables. Since version 1.4.23 the grouping page can also be used to filter and group by attributes

Step Properties Additional attributes for steps are displayed in the Details window and in the report. However, they cannot be used for filtering or grouping.

NI TestStand Plugin

Until version 1.4.3, only reports in the native IRS XML format could be used. Therefore the IRS XML TestStand plugin had to be installed to generate this specific format. Even though it still offers advantages for the measurement data analysis due to its compactness, this format is no longer a mandatory requirement!

Starting with version 1.5, the new NI TestStand plugin offers the possibility to directly read the standard formats NI XML, ATML5 and ATML6 used by TestStand!

The used format is automatically recognized by the plugin and the corresponding import type (XML / ATML5 / ATML6) is selected. If the file type is known, it can be set explicitly in the plugin settings, which can slightly speed up the import process.

Otherwise the handling does not differ from the IRS XML plugin.

Imported Executions (Panel)

The Imported Executions window displays all the imported executions. The table contains meta information such as the test time, the serial number and the overall result.

Viewing and grouping

• Double-click on an execution to open it in the Report Viewer and view the report. Measurement filters already applied are also taken into account.

• As already described in the section "Filtering", the executions can be filtered by properties. The set of all executions that match the filter criteria remains.

• Executions can be grouped in this view as well. To do so, drag the column header into the grouping area. However, this grouping only serves the current display and has no influence on the later necessary grouping.

Menu bar

In the menu bar an import filter can be created, loaded and applied.

Select and delete unwanted executions

It is often useful (e.g. due to aborted tests or during the commissioning phase) to remove certain executions from the data set before the data is analyzed.

• Press CTRL+A followed by DEL to remove all currently visible executions from memory. For example, a filter can be used to display all executions with status "Error" and "Terminated" and then remove them all at once.

• Single or multiple Executions can be removed from the table by left clicking (+ CTRL key for multiple selection) and pressing the DEL key

• It is not possible to load the same execution multiple times. Already loaded executions are recognized by their unique ID and skipped. Note: When reloading, the existing execution is NOT changed. Steps removed by measurement filters will not come back in this case! To actually reload an execution, it must be removed from the list first!

• Changes to the execution list lead to a recalculation of the results. However, already created groups remain as long as they contain at least 1 Execution. It is possible that deleting already created graphs will invalidate them if the measurements used no longer exist afterwards. Therefore groups should be created only after the import process is finished.

Export execution reports

To create an overall report for multiple test executions, select the desired rows and press "Export". A dialog will open with various options to show/hide specific content components. Configure the desired view and select the desired export format using the "Export" drop-down button:

Import Tips

Custom plugins

To integrate a custom plugin into the software, it must be placed in the subfolder 'Plugins' in the installation directory. After a restart it will be recognized automatically.

Import filter

When importing data into the Report Analyzer, all test steps of an execution are imported by default. During subsequent analysis, each imported test step is analyzed. Since the executions often contain considerably more steps than are of interest for the analysis, the executions can be reduced to the relevant test steps by so-called import filters.

The import filter is a list of measurement IDs. All steps whose measurement ID is found in the list are kept, the rest is discarded. The filter can be applied later to the loaded list above already during the loading process.

Advantages:

• The clarity is improved, irrelevant measurement steps are removed from the lists

• The processing speed for a large number of executions is increased

• Memory requirements are reduced (both in RAM and when saving data sets)

Downside:

• Accidentally removed steps can only be retrieved by removing the execution from the list and then re-importing

• The status of the execution is maintained. This can lead to inconsistent displays and interpretation problems, e.g. if the filter has removed all erroneous steps, but the overall result is still "Failed" or "Error".

A test step is always identified by its unique Measurement ID.

Create Import Filter

To create the import filter, select the Create Import-Filter button in the Import tab of the menu bar.

All available test steps are displayed on the left side of the window. By dragging and dropping, all test steps to be included in the filter are dragged to the right side of the window.

Using the input field, the list of test steps can be restricted to the search term entered:

After all test steps have been selected, the filter is saved by clicking the Export Filter button and the dialog can be closed.

Load import filter

The Open Import Filter button loads the import filter (but not yet applied). There are now two ways to use the filter:

Apply filter once

The previously loaded filter is applied by the Apply button. The program processes all previously imported executions and discards all test steps that are not listed in the import filter. Depending on the number of loaded executions this process can take some time. With this method the test executions are first loaded completely into memory and then the unneeded test steps are removed.

Apply filter during import

If the filter criteria are already known and the filter already exists, filtering can take place during the import process. For this purpose the button Active is selected before the import. Now the filter function is activated and when importing new executions the loaded filter is used. This filter method is recommended when importing large data sets where not all test steps are required but a large number of executions must be processed, e.g. for trend analysis. The working memory is already released during the import process.

YY Diagram

In the History & Statistics module, graphs for the same measurement from different groups can already be compared. The "Graph Comparison" module also offers the possibility to compare any measurements within the same group. This is helpful, for example, if the measurements are similar to each other, but were taken at different measurement points.

Procedure

1. Select the desired group and click on New YY Diagram

1. Rename the title if necessary:

1. Drag the desired measurement from the "ID" column in the graphs near the Y-axis When a green solid vertical bar appears, release the mouse button. The measurement series is added to the graph:

1. Repeat step 3 until all desired graphs are included in the diagram:

1. After adding three measurement series our example now looks like this

1. Each graph has an additional Y-axis. If all values are in a similar range, you should activate Single Y-Axis:

XY Diagram

For an X-Y diagram, proceed in the same way, but here one measurement is drawn to the X-axis and a second to the Y-axis. The resulting X/Y values are entered as points:

The similarity analysis shows linear relationships between at least two measurements. For this purpose the correlation coefficients are calculated according to Karl Pearson and displayed in a matrix. The correlation factors have values between -1.0 (direct negative correlation, red) and 1.0 (direct positive correlation, green). The cells are colored with a gradient based on their value. The stronger the correlation deviates from 0 (transparent), the stronger the cell is colored.

The correlation can be calculated for a selected group between selected or all measurements. The selected measurements are plotted as columns. For each existing measurement the factors are then calculated in one row. The selection therefore only affects the columns. The rows are always created over all measurements.

Export

• Export All Values: All visible results are written to the selected file in CSV format according to the settings

• Export Selected Values: Only the selected rows of the result table(!) are written to the selected file in CSV format according to the settings

The trend analysis can show whether a trend can be identified when a series of measurements is viewed over the long term. Since trends usually develop very slowly due to wear, material aging or mechanical effects, it is necessary to use either very large amounts of data or suitable samples over a longer period of time.

Settings (Panel)

In the settings, the desired group is set first. In the Test Step Selection section, you can optionally hide entries of failed tests to avoid falsifying the result by bad parts. Finally, the system influence shall be recognized here.

In the section Floating Mean the window size for the floating average is set. A minimum of 20 or 10% of the total number is recommended (e.g. 100 for 1000 measurements)

Afterwards, the parameters whose trend is to be analyzed are selected. A curve is calculated for each value selected here and displayed in the "Floating Mean" panel. Note: The window may be in the background by default.

• Calculate All: all available measurements are calculated

• Calculate Selected: only the upper part of selected measurements are calculated

For each calculation, one result row is entered in the "Trend Analysis" table. When clicking on the respective row, the corresponding graphs are displayed.

History (Panel) This graph shows the course of the measured values (raw data) as well as a linear interpolation and an interpolation with a polynomial of adjustable degree. From this, an overall trend can already be read off if necessary.

Floating Mean (Panel) This graph shows the smoothed course of selected metrics of the measurement. The window size of the moving average can be adjusted.

• The watchlist lists all entries and comments added via the corresponding watch function, see for example the comment function in .

• The comments are saved as Data Set when you save the file

• A double click on an entry (or click on the button "Show Report") opens the corresponding test report.

• Click on "Delete" to delete the selected entry.

App Settings

Here basic settings for the application are made, which can influence both the display and the calculation.

General

• Significant Digits: Sets the number of significant digits if numbers are rounded or formatted for display or in the report

• Use simplified group names: Concerns the automatic naming of groups. If activated, the name of the grouped column is omitted and only the value itself is used as name of the group, e.g. instead of "Product ID: XYZ" only "XYZ". Depending on the source of the grouping in the report, this will be easier to read. If the grouping is unclear, a descriptive name can also be helpful

• Use group name as default graph title: If activated, the "Graph Comparison" module uses the group name as default for the graph name. Otherwise the name remains empty and must be set by the user

Default Analysis Parameters

Defines the parameters for the calculation of some process parameters. When changes are made, the formulas are automatically updated to visualize the effects. See also the section Measurement Analysis Table!

• Histogram Groups: Sets the number of groups used for histogram generation. If "0" is entered here, the program will automatically calculate a reasonable number based on the number of measurement data

View Settings

The sections History and Statistics and Graph Comparison of the View Settings define preferences for the view in the respective modules.

Theme: Changes the display of the whole application (dark / light)

Graphs

• Visual Groups: Specifies the maximum number of groups that are visualized in the graph at the same time. If there are more groups, only the number set here will be displayed in graphs. This has no influence on the number of groups in the tables.

Export Settings

Here the default paths for various export files are set.

You can also define your own logo for the report, which will replace the IRS logo

This analysis method allows you to view the error distribution of a group and compare it with other groups.

The error distribution is always calculated in groups in two steps: 1st stage: The "Result" entry of an execution is considered, i.e. the overall result of an execution Step 2: The "Status" entry of a test step is considered, i.e. the partial result of an execution

The following figure shows the start page of the "Error Distribution" analysis method

The error distribution of the executions (1st level) of each group is displayed in a diagram. If you click on a diagram, you get to the Details view and get the additional information about the error distribution of the test steps (2nd level). In addition, the failures between the groups can then be compared.

Details

The detail view of the error distribution is structured as follows:

• On the left is the selection of the group to be viewed

• In the middle you see an overview of all test steps ("Measurements")

• Below you can switch between two panels: "Diagrams" and "Executions".

Measurements

This panel lists all test steps contained in the test executions of the respective group. The measurement ID, the step name and the description are displayed. It is also possible to see how often a step was performed and the status "Pass", "Fail", "Error" or "Terminated". This value is also set in relation to the total number of steps performed and displayed as a percentage value.

An important value for root cause analysis is the "First error" column or its percentage value. This indicates in how many test executions this measurement was the first failed step. With a filter > 0 on this column and descending sorting, it is very easy to create a list of the top causes of failure in each group:

Please note in this context: The first failed test step is not necessarily the step that led to the overall result or termination, even if this is normally the case. Similarly, "Failed" steps do not necessarily lead to a failed overall result of the test execution. This depends on the respective step settings and cannot be seen from the report.

The tables can be sorted and filtered, and conditional formatting can also be defined. Most of these settings are also used for the export.

When selecting a row in "Measurements", the affected test executions are displayed in the "Executions" table and the corresponding diagrams are updated.

Diagrams

This panel is divided into three areas. In the left area, a diagram is displayed which shows the status of all steps performed (as shown in the "Measurements"). In the middle, the results of all defined groups are compared in percent. Each group is displayed as a separate bar, so that a comparison across groups is quickly possible. If you move the mouse pointer over a bar, the group name and further information is displayed. The right area is identical to the middle area, but the values are shown in absolute numbers instead of percentages. This also allows you to quickly determine whether, for example, groups have the same workload or whether errors occur group-specifically or across groups.

Executions

If the operator is not only interested in the distribution but also wants to take a closer look at the logs, which are e.g. failed, he can do this in the panel "Executions". This panel lists all executed executions that have executed the selected test step. The operator gets various information displayed, such as the serial number. The test report can be opened and viewed by double clicking on a line. In the case of repeated measurements, it is thus possible to determine, for example, whether a failure occurs only with a single assembly or batch.

Export

There are three possibilities to export the analysis data of the error distribution:

1. Report Wizard

2. Table export

3. Windows clipboard

Report wizard

• Select the desired groups and measurements which should be included in the report:

• Click on preview

A preview is then automatically generated for each selected group, which contains the following information:

• Complete overview of all executions (1st part)

• Complete overview of all currently visible test steps per selected group (2nd part)

As soon as the preview is completely generated, the report can be saved in various formats. This can be done with the button "Export..." or the arrow below the button.

Report current view

The overview of all test steps is exported in the same way as the "Measurements" table is displayed in the Report Analyzer. I.e. filter, sorting and conditional formatting is taken 1:1 from the display.

Table Export

The user selects in the displayed dialog which groups and test steps are to be exported. This export method saves the table entries of the "Measurements" panel for each selected measurement line by line in a text file. This can then be opened in Excel, for example, and further processed as required. In addition, it can configure the following settings:

• File extension

• column separator

• Decimal point separator

• Output folder

• File name Click on "Export" to create the desired file.

Windows clipboard

In each table, the operator has the possibility to select cells or rows and copy them to the Windows clipboard by pressing CTRL+C. The data can then be pasted into a text file or Excel spreadsheet. With the shortcut CTRL+A the complete table can be selected. For example, the data can be exported from the panel "Executions".

The profiler is an alternative view of the test report and is displayed using the Report Viewer. The dialog opens by double-clicking on a single test execution in almost any suitable list (import, grouping, error distribution, etc.) You can access the profiler via the second tab at the top left:

Test steps from the report are displayed graphically in the order in which they occur, with the start time on the time axis. The length of the bar symbolizes the execution time of the test step. The colors have the following meaning:

• Blue: A test step with status "Done" or also a group total - Red: A test step with status "Failed" - Green: A test step with status "Passed" - Dark blue: A test step with status "Terminated" - Orange: A test step with status "Error" - Yellow: There is no data in the report for this section (gap)

The test steps are organized in groups according to the information in the report.

The functions of the toolbar in detail:

• Export: Provides the option to export the current view to various output formats. Note the page formatting options

• Expand: All groups and (non-filtered) steps are displayed

• Collapse: Only the group overviews are displayed

• Zoom out: Decrease current zoom level

• Zoom all: Sets the limits so that all contained test steps fit between the left and right margins (even those not currently visible)

• Follow: Align top entry to left margin, keep zoom (see below)

• Zoom Auto: Fit vertically visible area horizontally (see below)

• Zoom In: Increase current zoom level

• Show groups: steps in the same group are highlighted with a blue rectangle

• Show gaps: time periods not covered by the report are displayed as yellow pseudo-steps

• Show results: Shows the column with the step result

• Show value: Shows the column with the measured value - Show description: Shows the column with the step limits

• Show details: Shows the column with info text and additional results

• Min duration: The slider activates a filter for the minimum step length displayed (see below)

The view offers various tools to make navigation easier:

Follow

The entries are moved horizontally during vertical scrolling so that the first entry remains aligned at the top left. In contrast to 'Auto Zoom', the zoom level is retained. This mode is particularly suitable for optimally comparing execution times

Auto Zoom

The currently visible entries are fitted exactly into the available area so that the complete selected section is always visible. Please note that this can impair comparability because the zoom level is changed continuously.

Show Gaps

If this function is activated, the viewer creates pseudo entries for detected definition gaps between the end of the last step and the start of the next step. This can help to identify sections with longer execution times that would otherwise remain undetected due to missing entries in the report.

Minimum step duration

Note: The internally used IRS report does not save any information about the call hierarchy. The names of the sequences or sequence calls (depending on the configuration) are used as the "group". The position of the group in the sequence therefore says nothing about the call depth