Test Harness & Suites

Fresh

Test Harness XML

The Test Harness is the top-level container. Every app must have at least one harness file.

<TestHarness>
    <Name>Main Application Test Harness</Name>
    <TestSuiteFiles>
        <TestSuiteFile path="path/to/ui_tests.suite" />
        <TestSuiteFile path="path/to/api_tests.suite" />
    </TestSuiteFiles>
    <TestHarnessFiles>
        <TestHarnessFile path="path/to/another_harness.harness" />
    </TestHarnessFiles>
</TestHarness>

Element	Description
<Name>	Identifies the test harness
<TestSuiteFiles>	List of suite files to execute
<TestHarnessFiles>	Optional nested harnesses

Test Suite XML

Each suite groups test scripts for a specific app:

<TestSuite>
    <Name>UI Test Suite</Name>
    <AppName>com.example.myapp</AppName>
    <Description>User Interface tests</Description>
    <TestScripts>
        <TestScript prefix="LP">
            <Path>tests/ui/login_page_tests.py</Path>
        </TestScript>
        <TestScript prefix="SM">
            <Path>tests/ui/message_sending_tests.py</Path>
        </TestScript>
    </TestScripts>
</TestSuite>

Element	Description
<Name>	Suite identifier
<AppName>	Target application package name
<Description>	Suite purpose
<TestScript prefix>	Unique prefix for test case IDs
<Path>	Path to test script file

Test cases are identified as <Prefix>:<ID> (e.g., LP:1, SM:10).

Test Execution Configuration (TEC)

<TestExecutionConfiguration>
    <Target>
        <Type>Harness</Type>
        <FilePath>path/to/mainHarness.harness</FilePath>
    </Target>
    <LogConfiguration>
        <ReportLogPath>report.log</ReportLogPath>
        <ExecutionLogPath>test.log</ExecutionLogPath>
    </LogConfiguration>
    <StartTime>2024-04-10T08:00:00</StartTime>
    <Devices>
        <Device name="Pixel 4"/>
        <Device androidVersion="10" count="10"/>
        <Device groupName="group1"/>
    </Devices>
    <Iterations>5</Iterations>
    <TestRun>
        <Include>LP:1</Include>
        <Include>LP:2-4</Include>
        <Include>SM:10</Include>
    </TestRun>
</TestExecutionConfiguration>

TEC Elements

Element	Description
<Target>	Harness or Suite to execute
<LogConfiguration>	Report and execution log paths
<StartTime>	Scheduled execution time
<Devices>	Target devices (by name, version, group, count)
<Iterations>	Number of repetitions (default: 1)
<TestRun>	Selective test execution filter

Device Selection Rules

Specification	Behavior
name="Pixel 4"	First device matching name
androidVersion="10"	Match by Android version
count="10"	Number of devices needed
groupName="group1"	All devices in group
Empty <Device/>	Any available device

Constraint

groupName and count cannot coexist on the same <Device> element.

TestRun Filtering

Pattern	Runs
<Include>LP</Include>	All tests with LP prefix
<Include>LP:1</Include>	Only LP test case 1
<Include>LP:2-4</Include>	LP cases 2, 3, and 4
No <TestRun> defined	Entire harness runs

Output

When a TEC run completes, two files are created:

1. Result file (JSON) — test outcomes
2. Log file — execution details

The framework includes a primitive report generator. Custom report generators can be used to process the JSON results.