NI TestStand offers an excellent built-in report system. Using the reports, all relevant step details and parameters as well as the test sequence can be reproduced in detail at a later date if required.
While this file depth is helpful for debugging, the file size of this report makes it less suitable for archiving, and the complexity of the data structure makes it poorly suited for machine evaluation. Since practically any data type can occur, the extraction of the actually relevant measured values is sometimes tedious.
In production, however, it is primarily the measured values that are of interest, rather than the test sequence. The IRS XML Report was designed to provide a standardized data structure for the most frequently required data types and easy access to the measured values in a compact format. In this form, the data can then be easily analyzed by the IRS Report Anlayzer.
Compact format
Generic structure
Support for user-defined header data
Available as TestStand Report Plugin (contact IRS)
Storage as XML (if required also automatically packed in GZip format, approx. 90% space saving)
.NET API (.NET 4.6.1)
HTML preview
Direct support of custom stylesheets (XSL) for customized preview or special output format
Multi-instance capable
TestStand Report preview
Support for asynchronous processing
Standalone operation as assembly possible, e.g. for processing or generating reports from LabVIEW or other programming languages
Works in all plugin-enabled TestStand versions from 2013, 32-bit or 64-bit
Dynamic generation of the storage path using expressions based on report properties (e.g. serial number, date, status, custom fields)
Plugin callbacks for dynamic configuration on UUT basis (if required)
Configuration dialog for TestStand Report Options
Viewer program for direct opening with HTML view
CSV Writer as configurable extension for generating simple overviews in CSV format (requires IRS Report Plugin)
Data base for IRS Report Analyzer
Used by many IRS customers for over 10 years as standard format
IRS Reporting was not designed as a complete replacement or alternative for NI Reporting. Rather, its purpose is to provide a compact extract of measured values from a complex data structure as a flat list. To achieve this, some conceptual compromises are necessary, resulting in the following rules and limitations. However, these are essentially the same restrictions that apply to MES systems:
Supported data types for measured values:
Numeric data
String
Boolean
Arrays are not supported
The following contents are excluded from the report
Steps with status "Done" (exception: Additional Results were added)
Flow control steps
Sequence Calls. Only the calling step is suppressed. The included test steps are added to the list as a matter of course. The name of the sequence is noted in the report as "Group".
Custom step types that differ significantly from the standard steps in the data structure may not be recognized correctly
On-the-fly reporting is not supported
Basically the IRS Reporting works like the NI Reporting on the basis of the ResultList Arrays. Steps that are not included here cannot be processed.
Additional Result StepTypes(!) generate one step per Additional Result with the value of the added variable in the report (if the data type is supported). This can be used to provide arbitrary data for evaluation in the Report Analyzer, even if no (limit or pass/fail) test is associated with it.
Multiple Numeric Limit Tests generate one step per measurement. The name of the measurement is used as the measurement name. The Measurement ID gets an additional index
Additional Results in Tests (Numeric Limit Tests, Pass/Fail Tests, etc.) are added as attributes to the Step. These are listed e.g. in the HTML report as info text. An evaluation in the report manager, however, is not possible.
Since the data is largely transformed directly from the TestStand Report, the same rules apply to the assignment of limits as in TestStand: One-sided (LT/LE/GT/GE) or individual limits (EQ/NE) are ALWAYS written to the Lower Limit, regardless of the actual meaning. This rule is unfortunately system-dependent and must be considered if necessary with the representation! In the Report Viewer or the HTML Report Preview this assignment is corrected in the HTML Stylesheet. A corresponding demo stylesheet can be found in the examples. Note: For this reason the Reporting API contains an auxiliary class (NumericLimits), which calculates correct numeric values for upper and lower limit, tolerance, nominal value etc. from the step data using the comparison operator for all conceivable cases, as far as the situation permits.
In order to achieve the best possible result in terms of statistical evaluation, the following additional simple measures are strongly recommended. These have no influence on the test execution itself and are also compatible with TestStand Standard Reporting:
Each test step in the report is assigned a (ideally unique) Measurement ID. This allows the results of different test runs to be cumulated based on this ID, regardless of the position in the report. Since such an identification is unfortunately not provided as a TestStand feature, IRS has defined a special syntax convention for step names here. If the step name begins with a string enclosed in square or curly brackets, this string is used as the measurement ID. The structure of the ID is arbitrary regarding content and length. The following part is used as Measurement Name.
Examples
"[ADC_0200] Check ADC input value" =>. => Measurement ID: ADC_200 => Measurement Name: "Check ADC input value"
"{Setup CAN} Configure CAN Communication Interface" =>. => Measurement ID: Setup CAN => Measurement Name: "Configure CAN Communication"
In order to provide the report with more meta information than is possible via the built-in TestStand mechanisms, a special test step is inserted. The location of the step is irrelevant. It is only necessary to ensure that the data ends up in the ResultList. Therefore, the step cannot be used in callbacks such as PreUUT, where Result Recording is disabled by default.
Procedure:
Add a new step of type Additional Results. Recommendation: into the setup group of the MainSequence, if necessary also into the Cleanup, if during the test still further header data develops).
Rename the step to: REPORT HEADER DATA. The spelling (incl. upper/lower case and spaces) is important and must not differ, otherwise it will not be recognized!. Each Additional Result of this step is used as header entry. There are a number of predefined fixed fields, which thus get their own XML element in the report. All other entries that do not correspond to a reserved fixed field are translated as dynamic header attributes (XML element "Attr") in the report.
Available predefined (fixed) field names:
OrderId
ProductId
ProductType
TestMode
TestType
TestTitle
TestAuthor
TestRevision
SwRevision
HwRevision
CarrierId
AdapterId
FixtureId
PanelId
DataCode
Logfile
Comment
The spelling is important. Otherwise, in case of deviations, the reserved field is not used, but a similarly named attribute is created.
Important: The following data is read directly from the TestStand environment and therefore need not (or should not) be created as Additional Results:
Execution ID (GUID, wird ggf. vom System generiert oder aus dem ATML Report übernommen)
Startzeit
Ausführungszeit
Result (Status / Ergebnis)
UUT SerialNr
UUT PartNr
TestSocket Nr
Batch SerialNr
Station Id
User
Testplan (Name der Sequenzdatei mit dem Einsprungspunkt der Execution)
Infotext
Error Code
Error Message