Manual launcher (manual.jar)

The manual testing launcher can be used as is to perform manual testing if your tests are currently not automated and consist in just following some procedures. The launcher will use the testplan as entered in the system to request the user to perform some manual tasks and to answer some questions.


Configuration

The manual.xml file allows pre-configuring the launcher with some default values:

Parameter Description
General > set testcase as failed as soon as one step fails If set to true, when one step fails in the testcase the complete testcase is set as failed and the remaining steps are skipped.

Default value is: false
Timing > delay between testcases (ms) Default value is: 0


These values can be changed while creating the campaign session from XStudio.


Control Bar

The manual launcher includes a control bar allowing controlling the execution of the tests. The control bar includes the following buttons:
  • Go back to previous test
  • Go back to previous testcase
  • Restart current testcase
  • Pause current testcase
  • Resume current testcase
  • Go forward to next testcase
  • Go forward to next test


Timeout per test

While executing manual tests, the launcher provides instructions to the operator to execute actions (and do verifications). The tests are configurable so that some timing restrictions can be added. For instance, it is possible to give 10 minutes maximum for the operator to execute the actions/verifications (i.e. you need to test that a message appears on the screen of the SUT in less than 10 minutes; if the operator doesn't validate that he saw that message within the 10 minutes then the test is automatically set as failed).

If you want to set some specific timeouts to a test you just need to associate to this test one or several of the following attributes (with a value for the timeout):
  • timeout test description (ms): maximum delay to validate the test description
  • timeout test prerequisites (ms): maximum delay to validate the test prerequisites
  • timeout testcases execution (ms): maximum delay to execute any actions or verifications implied by the testcases in this test


When manually executed, each test will be applied with their specific timeout values.

Note also, that all these attributes are dynamic. This means that when creating the campaign session, you can overwrite any of these default values (on each test if necessary) to customize one campaign session execution:

Tutorial: Creating and executing manual tests

In this tutorial we will learn how to setup from scratch a manual test suite, execute it and analyze the results.


Declare the agent and the SUT

If it does not already exist, create an agent with the exact name (or the IP Address) of your PC. To know the name of your PC, use the ipconfig /all command on Windows and ifconfig command on Linux. This name/IP address must be ping-able from the network.
  • in the Agents tree, select the root folder
  • on the right panel, click on the create folder button
  • enter the name of the folder (i.e. “France Office”) and submit
  • in the tree, select the newly added folder (i.e. “France Office”)
  • on the right panel, click on the create agent button
  • enter the hostname or the static ip address of your local host and submit

Then create a SUT (System Under Test):
  • in the SUTs tree, select the root folder
  • on the right panel, click on the create company button
  • enter the name of the company (i.e. “Acme”) and submit
  • in the tree, select the newly added company (i.e. “Acme”)
  • on the right panel, click on the create folder button
  • enter the name of the folder (i.e. “QA”) and submit
  • in the tree, select the newly added folder (i.e. “QA”)
  • on the right panel, click on the create sut button
  • enter the name (i.e. “My Browser”) and the version (i.e. “1.0”) of the SUT and submit


Create a dedicated category for manual tests

We now need to create a category in the test tree that will gather all the manual tests.
  • in the Tests tree, select the root folder
  • on the right panel, click on the create category button
  • a dialog box is displayed
  • enter the name of the category (i.e. “Manual tests”).
  • enter the description of the category. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted.
  • enter the launcher to be associated with that category: manual.jar
  • click on submit


Create a test

To create a test:
  • in the tree, select the newly added category (i.e. “Manual tests”)
  • on the right panel, click on the create folder button
  • enter the name of the folder (i.e. “My browser tests”) and submit
  • in the tree, select the newly added folder (i.e. “My browser tests”)
  • on the right panel, click on the create test button
  • a dialog box including three tabs is displayed
  • fill the Details tab with the name (i.e. “basic test”) and priority of the test (leave the canonical path blank).
  • fill the Testplan tab with the prerequisites and the general description of the test. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted and submit




To create a testcase:
  • in the tree, select a newly added test (i.e. “basic test”)
  • on the right panel, click on the create testcase button
  • a dialog box including two tabs is displayed


  • fill the Details tab with the index (defining the order in which the testcases will be executed within a test), name (i.e. “first test case”) and general description of the testcase. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted. Do not forget to check the Implemented checkbox. Non-implemented test are NOT executed when running a campaign. In case of manual test, always check the Implemented checkbox.
  • select the Testplans tab and define all the steps and checks that will be needed in this testcase:
    • add a step:
      • in the tree, select the root folder
      • click on the create step button
      • enter the description of the step and submit
    • add parameters (opt):
      • in the tree, select the parameters node
      • click on the create parameter button
      • enter the description of the parameter and submit
      • repeat the operation if you need to specify more parameters
    • add checks (opt.):
      • in the tree, select the checks node
      • click on one of the create boolean operators buttons
      • click on the new operator and click on the create check button
      • enter the description of the check and submit
      • repeat the operation if you need to specify more checks (you can mix as many different boolean operators as you want)
    • repeat the operation for every step and submit
  • click on submit
  • immediately, the testcase appears in the sub-tree on the left panel

You can create other testcases and other tests using the same procedure.


Run an individual test

  • In the test tree, select the newly created test (i.e. “basic test”) (not the test case!) and press the "start" button
  • Check in the Agent tab of the pop up window that your PC is selected by default
  • Select the "My browser 1.0" in the SUT tree
  • Select a pre-existing configuration in the Configuration tab
  • If no configuration are available:
    • click on the create configuration button
    • a dialog box is displayed
    • enter the name of the configuration
      • fill in all the forms displayed and submit
  • Press the Submit button



Some popup dialog boxes will ask the operator to execute some operations and/or verify some assertions





  • When the test is finished, you can control the results in the campaign tree (a new test campaign and campaign session have been automatically created in the campaign tree)
  • In the content tab, you can select a test, then a test case (in the dynamic sub-tree) and check the complete log of the execution



Creating a test campaign

At this stage you know how to individually run a manual test. But how to deal with a complete testsuite? XStudio provides a very efficient and simple way of managing manual test campaign. For this, two different elements must be distinguished: test campaign and campaign session:
  • The test campaign defines just a list of tests to be executed
  • A new campaign session will be created each time the campaign will be performed. It adds to the test campaign notion, the target on which the tests will be performed and how. The "how" is characterized by some dynamic configurable parameters.

To create a campaign:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create campaign icon
  • a dialog box including two tabs is displayed
  • fill the Details tab with the name of the campaign
  • in the Content tab, select all the tests you want to be part of this campaign
  • click on submit
  • immediately, the campaign appears in the tree
To create a campaign session:
  • in the tree, select a campaign
  • on the right panel, click on the create campaign session button
  • a dialog box including seven tabs is displayed
  • fill the Details tab with the name of the session.
  • (opt.) select the Test operator tab and pick the user who will run the session
  • select the Agent tab and pick the agent on which the session will be run. If you wish to run the tests on the local host, leave the default selection (that should already match the local host)
  • select the SUT tab and pick the SUT on which the session will be run
  • select the Configuration tab and pick the configuration you which for each category involved in this campaign session. Once a campaign session is created, it is impossible (by purpose) to change the configurations associated.
  • (opt.) if needed, modify the dynamic attributes values in the Attributes tab
  • (opt.) select the CC Emails tab and check some users to receive a notification when the campaign session will be completed
  • click on submit
  • immediately, the campaign session appears in the tree in idle state


Run a campaign session

To run a campaign session:
  • in the tree, select a campaign session
  • on the right panel, click on the start button
  • immediately, the Test campaign details screen appears and will display real time the results of testing