Create a VERA Test Set

Owned by HB Farris
May 12, 2022
3 min read


Figure: Create a Test Set, Associate it to a Tosca Execution List
In Tosca, create an Execution List. The Execution List contains any number of Tosca Test cases to run in order to achieve a test objective. The same concept exists in ALM and is called a Test Set. In both ALM and Tosca, a Test Case can be included in any number of Execution Lists.

In a VERA-Tosca Test Set, execution results are recorded at the Test Set/Execution List level. This is different from Manual ALM Tests that each execute at the Test level.
VERA-Tosca Test Set creation differs somewhat from Manual Test Set creation. With Tosca Test Sets, an Execution List is read from Tosca and a VERA Test Set is automatically created, along with its contained Test Instances, mirroring what is in the Tosca Execution List. With VERA-Tosca Test Sets, there is no ability to drag tests, from the Test Plan module, into a Test Set to build a list of Test Instances – the Test Sets are built from Tosca Execution Lists.

IMPORTANT: a VERA Test must exist in ALM's Test Plan module before the VERA Tosca Test Set can be created containing that test reference.

Preparation


Best Practice:
Before creating a VERA-Tosca Test Set and associating it with a Tosca Execution List, fully test the Execution List, verifying it runs successfully on target Tosca Agent machines. Then, set the ExecListWorkstate field to "COMPLETED".

To prepare for controlling a Test Set from VERA so that VERA can run and collect results for formal review/approval, perform the following steps:

  1. Make sure included tests abide by Best Practices
  2. Develop the Execution List and make sure it runs on your local desktop.
  3. Make sure it runs repeatedly, over time (i.e. it runs repeatedly today, it runs tomorrow, and the next day)
  4. Make sure it runs on target Tosca Agent machines.
    1. Allow time to configure, setup, and verify this step – especially if the Execution List is one of the first to run against a new System Under Test.
    2. If just setting up target machines for a project, log on to each. Answer first time Windows prompts such as asking about security choices in Internet Explorer. In Tosca Commander, run a representative Execution List for the target project.
    3. Make sure the Agent machine has: (See the "Tosca Remote Agents" section below)
      1. all required software, files, access to data, etc.
      2. a Workspace configured for the target project
      3. a parameter configured identifying the machine as able to run the project
      4. its status indicator in the System Tray showing as green – any other color indicates that a script may not run remotely on the machine.
    4. From Commander, on your local desktop, create a Test Event, drag the Execution List into it, along with a Configuration, and verify it remotely executes
  5. Mark the ExecutionList as "COMPLETED" by selecting that value in the ExecListWorkstate field.

Steps to Create a Test Set

Once verified that the Execution List runs successfully in Tosca, and its ExecListWorkstate field is set to "COMPLETED", log into ALM and create a Tosca Test Set by performing these steps:

  1. Select the target Test Lab folder and click the Tosca tool bar button. The Tosca Action Menu displays.
  2. Click 'Create Test Set'. If this button is disabled, hover the move over it to see why.
  3. Enter Tosca credentials, if prompted. In the selection dialog, select the Execution List to associate with the VERA Test Set. The Test Set is created along with Test Instances for each Test in the Execution List.
  4. Click the ALM 'Refresh' button.

Due to ALM technical limitations, a screen refresh is required, to see updates. Click 'Refresh' to update the screen


The Test Set status is set to 'Draft' and it is now associated with the Execution List in Tosca. Additionally, its name and Description field values are copied from the Execution List's Name and Description field.

  • Before running a Test Set, make sure that, in Tosca, there is a Test Event by the same name in the closest "TestEvents" root folder.


In order to execute/run the Test Set its status must be set appropriately. Typically, a Test Set simply requires its status to be "Ready for Execution". Some VERA environments are configured to pre-approve Test Sets. Either way, use the VERA "Go" toolbar button to change the Test Set status in order to allow Test Set execution. VERA's "Start Test Set(s)" operation changes the status of a Test Set to "Ready for Execution".