Manual

IRS Reporting

NI TestStand offers an excellent built-in report system. Using the reports, all relevant step details and parameters as well as the test sequence can be reproduced in detail at a later date if required.

While this file depth is helpful for debugging, the file size of this report makes it less suitable for archiving, and the complexity of the data structure makes it poorly suited for machine evaluation. Since practically any data type can occur, the extraction of the actually relevant measured values is sometimes tedious.

In production, however, it is primarily the measured values that are of interest, rather than the test sequence. The IRS XML Report was designed to provide a standardized data structure for the most frequently required data types and easy access to the measured values in a compact format. In this form, the data can then be easily analyzed by the IRS Report Anlayzer.

Features

  • Compact format

  • Generic structure

  • Support for user-defined header data

  • Available as TestStand Report Plugin (contact IRS)

  • Storage as XML (if required also automatically packed in GZip format, approx. 90% space saving)

  • .NET API (.NET 4.6.1)

  • HTML preview

  • Direct support of custom stylesheets (XSL) for customized preview or special output format

  • Multi-instance capable

  • TestStand Report preview

  • Support for asynchronous processing

  • Standalone operation as assembly possible, e.g. for processing or generating reports from LabVIEW or other programming languages

  • Works in all plugin-enabled TestStand versions from 2013, 32-bit or 64-bit

  • Dynamic generation of the storage path using expressions based on report properties (e.g. serial number, date, status, custom fields)

  • Plugin callbacks for dynamic configuration on UUT basis (if required)

  • Configuration dialog for TestStand Report Options

  • Viewer program for direct opening with HTML view

  • CSV Writer as configurable extension for generating simple overviews in CSV format (requires IRS Report Plugin)

  • Data base for IRS Report Analyzer

  • Used by many IRS customers for over 10 years as standard format

Limitations

IRS Reporting was not designed as a complete replacement or alternative for NI Reporting. Rather, its purpose is to provide a compact extract of measured values from a complex data structure as a flat list. To achieve this, some conceptual compromises are necessary, resulting in the following rules and limitations. However, these are essentially the same restrictions that apply to MES systems:

  • Supported data types for measured values:

    • Numeric data

    • String

    • Boolean

  • Arrays are not supported

  • The following contents are excluded from the report

    • Steps with status "Done" (exception: Additional Results were added)

    • Flow control steps

    • Sequence Calls. Only the calling step is suppressed. The included test steps are added to the list as a matter of course. The name of the sequence is noted in the report as "Group".

  • Custom step types that differ significantly from the standard steps in the data structure may not be recognized correctly

  • On-the-fly reporting is not supported

Basically the IRS Reporting works like the NI Reporting on the basis of the ResultList Arrays. Steps that are not included here cannot be processed.

Rules

  • Additional Result StepTypes(!) generate one step per Additional Result with the value of the added variable in the report (if the data type is supported). This can be used to provide arbitrary data for evaluation in the Report Analyzer, even if no (limit or pass/fail) test is associated with it.

  • Multiple Numeric Limit Tests generate one step per measurement. The name of the measurement is used as the measurement name. The Measurement ID gets an additional index

  • Additional Results in Tests (Numeric Limit Tests, Pass/Fail Tests, etc.) are added as attributes to the Step. These are listed e.g. in the HTML report as info text. An evaluation in the report manager, however, is not possible.

  • Since the data is largely transformed directly from the TestStand Report, the same rules apply to the assignment of limits as in TestStand: One-sided (LT/LE/GT/GE) or individual limits (EQ/NE) are ALWAYS written to the Lower Limit, regardless of the actual meaning. This rule is unfortunately system-dependent and must be considered if necessary with the representation! In the Report Viewer or the HTML Report Preview this assignment is corrected in the HTML Stylesheet. A corresponding demo stylesheet can be found in the examples. Note: For this reason the Reporting API contains an auxiliary class (NumericLimits), which calculates correct numeric values for upper and lower limit, tolerance, nominal value etc. from the step data using the comparison operator for all conceivable cases, as far as the situation permits.

Conventions

In order to achieve the best possible result in terms of statistical evaluation, the following additional simple measures are strongly recommended. These have no influence on the test execution itself and are also compatible with TestStand Standard Reporting:

Use of Measurement IDs

Each test step in the report is assigned a (ideally unique) Measurement ID. This allows the results of different test runs to be cumulated based on this ID, regardless of the position in the report. Since such an identification is unfortunately not provided as a TestStand feature, IRS has defined a special syntax convention for step names here. If the step name begins with a string enclosed in square or curly brackets, this string is used as the measurement ID. The structure of the ID is arbitrary regarding content and length. The following part is used as Measurement Name.

Examples

"[ADC_0200] Check ADC input value" =>. => Measurement ID: ADC_200 => Measurement Name: "Check ADC input value"

"{Setup CAN} Configure CAN Communication Interface" =>. => Measurement ID: Setup CAN => Measurement Name: "Configure CAN Communication"

Using REPORT HEADER DATA

In order to provide the report with more meta information than is possible via the built-in TestStand mechanisms, a special test step is inserted. The location of the step is irrelevant. It is only necessary to ensure that the data ends up in the ResultList. Therefore, the step cannot be used in callbacks such as PreUUT, where Result Recording is disabled by default.

Procedure:

  • Add a new step of type Additional Results. Recommendation: into the setup group of the MainSequence, if necessary also into the Cleanup, if during the test still further header data develops).

  • Rename the step to: REPORT HEADER DATA. The spelling (incl. upper/lower case and spaces) is important and must not differ, otherwise it will not be recognized!. Each Additional Result of this step is used as header entry. There are a number of predefined fixed fields, which thus get their own XML element in the report. All other entries that do not correspond to a reserved fixed field are translated as dynamic header attributes (XML element "Attr") in the report.

Available predefined (fixed) field names:

  • OrderId

  • ProductId

  • ProductType

  • TestMode

  • TestType

  • TestTitle

  • TestAuthor

  • TestRevision

  • SwRevision

  • HwRevision

  • CarrierId

  • AdapterId

  • FixtureId

  • PanelId

  • DataCode

  • Logfile

  • Comment

The spelling is important. Otherwise, in case of deviations, the reserved field is not used, but a similarly named attribute is created.

Important: The following data is read directly from the TestStand environment and therefore need not (or should not) be created as Additional Results:

  • Execution ID (GUID, wird ggf. vom System generiert oder aus dem ATML Report übernommen)

  • Startzeit

  • Ausführungszeit

  • Result (Status / Ergebnis)

  • UUT SerialNr

  • UUT PartNr

  • TestSocket Nr

  • Batch SerialNr

  • Station Id

  • User

  • Testplan (Name der Sequenzdatei mit dem Einsprungspunkt der Execution)

  • Infotext

  • Error Code

  • Error Message

Appendix

IRS Report Format (Example)

HTML Appearence / Viewer

Corresponding XML File

<TestReport xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" id="cb2b0bbb-7837-4007-b86a-b2ec277cc521" source="NI TestStand" converter="irs_xmlresult.xsl" version="1.5">
  <FileFormat>1.0</FileFormat>
  <FileType>IRS XML Test Report</FileType>
  <CreationTime>2019-05-09T14:07:27.4951931+02:00</CreationTime>
  <CreationHost>STAT-D</CreationHost>
  <User>Tester</User>
  <DllVersion>1.45.36.0</DllVersion>
  <TestExecution start="42.3787865" time="23.8706028" id="51c7d683-8d35-4703-b881-57ae50aa769c" ts="2019-05-09T14:07:03">
    <Result>Failed</Result>
    <SerialNr>IRSN190508200001</SerialNr>
    <SocketNr>-1</SocketNr>
    <BatchNr />
    <StationId>STAT-D</StationId>
    <User>operator</User>
    <TestPlan>D:\Testsystem\ECB-Demo.seq</TestPlan>
    <Infotext />
    <ErrorCode>0</ErrorCode>
    <ErrorMsg />
    <ProductId>8XC7518-2JP8_-____</ProductId>
    <ProductType>ECB350</ProductType>
    <TestMode>Standard</TestMode>
    <TestType>EOL</TestType>
    <TestTitle>Demo Report</TestTitle>
    <TestAuthor>Stefano Serrano / IRS</TestAuthor>
    <TestRevision>0.5.0.5</TestRevision>
    <SwRevision>V4.2.0.0</SwRevision>
    <HwRevision>12343.0</HwRevision>
    <CarrierId>2</CarrierId>
    <AdapterId>63A</AdapterId>
    <PartNr>8XC7518-2JP8_-____</PartNr>
    <DataCode>XY8XC7518-2JP8_-____###+IRSN190508200001</DataCode>
    <Attr name="Nominal Current" type="String" value="63.0A" />
    <Attr name="Nominal Frequency" type="String" value="50.0Hz" />
    <Attr name="Ground Fault" type="Boolean" value="False" />
    <Attr name="Number of poles" type="Number" value="3" />
    <Attr name="External N (type)" type="Number" value="0" />
    <Attr name="Breaker Type" type="String" value="2" />
    <Attr name="Protection Function" type="String" value="HN" />
    <TestSteps>
      <TestStep measid="RESM_4100" measname="Resistance L1" status="Passed" datatype="Number" limlo="100.000" value="360.147" limhi="2500.000" comp="GELE" unit="uOhm" stepname="[RESM_4100] Check L1 Resistence" steptype="NI_LimitMeasurement" info="" start="50.5258012" time="0.0157013" stepid="ID#:OgNqkZFG6RGN9ggAJw99EB" group="[RESM] Resistance Measurement" />
      <TestStep measid="RESM_4200" measname="Resistance L2" status="Passed" datatype="Number" limlo="100.000" value="416.869" limhi="2500.000" comp="GELE" unit="uOhm" stepname="[RESM_4200] Check L2 Resistence" steptype="NI_LimitMeasurement" info="" start="50.5420167" time="0.0001318" stepid="ID#:UKPbOJJG6RGN9ggAJw99EB" group="[RESM] Resistance Measurement" />
      <TestStep measid="RESM_4300" measname="Resistance L3" status="Passed" datatype="Number" limlo="100.000" value="376.531" limhi="2500.000" comp="GELE" unit="uOhm" stepname="[RESM_4300] Check L3 Resistence" steptype="NI_LimitMeasurement" info="" start="50.5425627" time="0.0001225" stepid="ID#:UaPbOJJG6RGN9ggAJw99EB" group="[RESM] Resistance Measurement" />
      <TestStep measid="CCTM_0140" measname="Current" status="Passed" datatype="Number" limlo="59.85" value="62.93" limhi="66.15" comp="GELE" unit="A" stepname="[CCTM_0140] Measure and validate current from PA" steptype="NI_LimitMeasurement" info="" start="64.6433575" time="0.6369838" stepid="ID#:RhMyrTEa6RGWU0xSYhBZqD" group="[CCTM] Combined CurrentTransformer Module Test" />
      <TestStep measid="CCTM_0200" measname="Measure CCT secondary current" status="Failed" datatype="Number" limlo="190.0" value="179.2" limhi="210.0" comp="GELE" unit="mA" stepname="[CCTM_0200] Measure CCT secondary current" steptype="NumericLimitTest" info="" start="65.2808703" time="0.3575013" stepid="ID#:/HuUJEUa6RGWU0xSYhBZqD" group="[CCTM] Combined CurrentTransformer Module Test" />
    </TestSteps>
  </TestExecution>
</TestReport>
PreviousInstaller

Last updated

Was this helpful?