User Tree

Here is a typical user tree. It immediately shows a number of useful information:
  • the total number of users
  • the number of users in each company
  • the number of users in each folder
  • the status of the users (enabled/disabled indicated by the color of the icon)


Hierarchies, working groups and teams can be easily managed in XStudio by creating a structure of folders hosting users.


Create a company

This first thing to do is to create your own company. To do this:
  • in the tree, select the root folder
  • on the right panel, click on the create company button
  • enter the name of the company and submit
  • immediately, the company appears in the tree
Note: this company entity will be automatically shared in the SUT tree.


Create a hierarchy in the company

Then, it’s important to well define the working groups and teams in the company. This can be achieved by creating a complete tree of folders and sub-folders. To do this:
  • in the tree, select a company
  • on the right panel, click on the create folder button
  • enter the name of the folder and submit
  • immediately, the folder appears in the tree
Of course, a sub-folder can be created into a folder. Redo the same operation until you are satisfied with the organisation.


Create a user

Now that you have a company with its internal organization defined, you’re ready to create some users. To do so:
  • in the tree, select a folder
  • on the right panel, click on the create user button
  • a dialog box including two tabs is displayed
    •  
  • fill the Details tab with the user name, password, preferred language, email address and location. Do not forget to check the Enabled checkbox. A disabled user cannot login into the system.
  • the Rights tab is here to select what this user will be able to do. For now, just check the first checkbox corresponding to the root folder (this will grant automatically ALL the rights to this user)
  • click on submit
  • immediately, the user appears in the tree
From now, you can exit XStudio, restart and login with the new user credentials.


Submit a new absence

Each user can enter some absences. After submission, absences have the status new. Once the manager has set them as approved, the status changes accordingly. User’s absences are visible by selecting the Absences tab:



To enter a new absence:
  • in the tree, select the user
  • on the right panel, select the Absences tab
  • press the create absence button
  • enter the type, start and stop dates and optionally a comment and submit
  • immediately, the absence appears in the list




Edit an absence

If you are manager (have the rights to approve absences), then you can edit an absence and change its status. To edit an absence:
  • in the tree, select the user
  • on the right panel, select the Absences tab
  • select an absence
  • press the edit absence button
  • enter the type, start and stop dates and optionally a comment and submit
  • immediately, the absence appears in the list




Check the calendars

  • in the tree, select the root folder or a user
  • on the right panel, select the Calendar tab
  • the calendar is displayed. Two modes are available:
    • Monthly mode
    • Half-yearly mode
  • You can select the mode by clicking on the appropriate toggle button:
    • Monthly:
    • Half-yearly:
  • You can also “move” the calendar by using the arrow keys.






Changing admin password

 Note: The default admin user is configured with the password “password”. This is strongly recommended to change this password as soon as possible. To do so:
  • in the tree, select the admin user (XQual/Administration/admin)
  • on the right panel (Details sub-tab), enter the new password, confirm and submit



SUT Tree

Here is a typical SUT tree. It immediately shows a number of useful information:
  • the total number of SUTs
  • the number of SUTs in each company
  • the number of SUTs in each folder
  • the version of each SUT




A SUT is an abstract object representing the target we want to test. It can be a hardware device or a software component. The SUT must be detailed enough so that we can identify it easily.A SUT should be uniquely defined through its name and version.
It is possible to group/organize the SUTs by creating a structure of folders and sub-folders hosting the SUTs.


Create a SUT

Here is the process to create a new SUT:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create sut button
  • enter the name and the version of the SUT and submit
  • immediately, the SUT appears in the tree



Create a SUT Inheriting requirements from another SUT

Here is the process to create a new SUT inheriting requirements from another SUT:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create sut button
  • enter the name and the version of the SUT
  • select the Requirements tab
  • click the Preset import from SUT settings button
  • select the reference SUT and submit
  • click the Preset import from SUT button
  • (opt.) select/unselect some requirements if required
  • submit
  • immediately, the SUT appears in the tree




Agent Tree

Here is a typical requirement tree. It immediately shows a number of useful information:
  • the total number of agents
  • the number of agents in each folder
  • the operating system of each agent




All hosts on the network that will have XStudio of XAgent installed on MUST be included in the tree to be able to execute some tests.


Add my local host

Here is the process to add your local host to the system:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create agent button
  • enter the hostname or the static ip address of your local host and submit
  • immediately, the agent appears in the tree
 Note: what you enter in the Name field MUST be ping-able. To check this, open a windows console and ping it.


Add all necessary hosts

Redo the same for all hosts that will have XStudio or XAgent installed.



Requirement Tree

Here is a typical requirement tree. It immediately shows a number of useful information:
  • the total number of requirements
  • the number of requirements in each category
  • the number of requirements in each folder
  • the status of each requirement (indicated by the color of the icon)
  • the priority of each requirement (indicated by the column)






Requirements are all the conditions the SUT should be compliant with. Generally, the requirements list is the first thing we do before working on detailed specification. In a perfect world, the SUT should come with a list of requirements but this may not be the case.

Entering the requirements of the SUT is an optional task. We do encourage to completely defining the requirements though. This information is very useful for coverage reporting.

You can add in the requirement tree the complete description of each requirement or you can decide to just point to the external requirements document(s).


Create a category

This first thing to do is to create a category. To do this:
  • in the tree, select the root folder
  • on the right panel, click on the create category button
  • a dialog box is displayed
  • enter the name of the category.
  • enter the description of the category. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted.
  • enter the launcher to be associated with that category: manual.jar indicates that all the tests that will be included under that category will be executed using the manual test launcher.
  • click on submit
  • immediately, the category appears in the tree

 Note: this category entity will be automatically shared in the Specification, Tests and Defect trees.


Create a requirement

Now that you have a category with its internal organization defined (see, you’re ready to create some requirements. To do so:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create requirement button
  • a dialog box is displayed



  • pick a requirement type
  • if required, pick a requirement category
  • enter the name of the requirement
  • enter the description of the requirement. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted.
  • select the status of the requirement (at creation time, you can only choose New or Ack)
  • select the priority of the requirement
  • click on submit
  • immediately, the requirement appears in the tree




Edit a requirement

The requirements will probably be edited several times by different people and will go through a complete workflow including three states:

Overlay Represents
New New requirement
Ack The requirement is being reviewed by the person who is supposed to sign it off
Approved The requirement has been approved



To edit a requirement:
  • in the tree, select a requirement
  • on the right panel (details tab), edit the information you wish and submit

Note: Depending of the rights you’ve been granted, you may or may not be able to set the status to a specific state.



Specification Tree

Here is a typical specification tree. It immediately shows a number of useful information:
  • the total number of specifications
  • the number of specifications in each category
  • the number of specifications in each folder
  • the status of each specification (indicated by the color of the icon)
  • the priority of each specification (indicated by the column)



Specifications are detailed and unitary description of a specific behavior.

You can add in the specification tree the complete description of each specification or you can decide to just point to the external requirements document(s).


Create a specification

To create a specification:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create specification button
  • a dialog box is displayed



  • check the formal checkbox if needed
  • enter the name of the specification
  • enter the description of the specification. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted.
  • select the status of the specification (at creation time, you can only choose New or Ack)
  • select the priority of the specification
  • click on submit
  • immediately, the specification appears in the tree

Edit a specification

The specifications will probably be edited several times by different people and will go through a complete workflow including three states:

Overlay Represents
New New specification
Ack The specification is being reviewed by the person who is supposed to sign it off
Approved The specification has been approved



To edit a specification:
  • In the tree, select a specification
  • on the right panel (details tab), edit the information you wish and submit

Note: Depending of the rights you’ve been granted, you may or may not be able to set the status to a specific state.



Project Tree

Here is a typical project tree. It immediately shows a number of useful information:
  • the total number of projects
  • the total number of sprints
  • the total number of tasks
  • the number of sprints in each project
  • the number of tasks in each project
  • the status of each sprint (indicated by the color of the project icon)
  • if the tasks have been already affected to a sprint or not (indicated by the color of the task icon – a grayed icon indicates that the task is already affected to a sprint)






A project must be created for any new “product” you want to deliver. A project is usually a development project but can also be more generic. Project can be created for the development of the main product line but also for automated testsuites and much other internal project in your company.

A project is made of tasks. Most of the time, a task is the smaller entity that a developer can develop (a feature is made of several tasks).

The scrum methodology defines the notion of sprints. A sprint is the result of any iteration in a project. So, to deliver a product, you will deliver several intermediate releases, each corresponding to a sprint. A sprint is generally at most 2 or 3 weeks long. A number of tasks will be associated to each sprint and at the end of the sprint a demo of all the features developed can be done.


Create a project

To create a project:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create project button
  • a dialog box is displayed
  • enter the name of the project
  • enter a focus ratio (this corresponds to the percentage of time that people will spend on effective work)
  • enter the description of the project. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted.
  • click on submit
  • immediately, the project appears in the tree

Create a task

To create a task:
  • in the tree, select a folder belonging to a project (create one if necessary)
  • on the right panel, click on the create task button
  • a dialog box is displayed
  • enter the name of the task
  • enter the description of the task. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted.
  • enter the priority of this task
  • enter the estimated effort in man.days (people working 100% of their time)
  • click on submit
  • immediately, the project appears in the tree

Create a sprint

To create a sprint:
  • in the tree, select a project
  • on the right panel, click on the create sprint button
  • a dialog box is displayed
  • enter the name of the sprint
  • enter the description of the sprint. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted.
  • select the status of the sprint (at creation time, you can only choose Idle or Running)
  • select the start and stop dates for this sprint
  • click on submit
  • immediately, the project appears in the tree

Edit a sprint

The sprints will probably be edited several times by different people and will go through a complete workflow including three states:

Overlay Represents
Idle Idle sprint
Running The sprint is currently running
Finished The sprint if finished



To edit a sprint:
  • In the tree, select a sprint
  • on the right panel (details tab), edit the information you wish and submit
  • the duration (in effective man.days) is automatically updated

Note: Depending of the rights you’ve been granted, you may or may not be able to set the status to a specific state.


Allocate some resources to a sprint

All the tasks associated to a sprint will be performed by a pool of resources. These resources are users (taken from the user tree). To allocate some resources to a sprint:
  • in the tree, select the sprint
  • on the right panel, select the Resources tab
  • un-toggle the select filter button to display the complete user tree
  • check all the users that need to be allocated to the current sprint and indicate by editing the percentage of availability of each resource
  • (opt.) re-toggle the select filter button to display only the selected users
  • click on submit

Note: You can check if some of the resources are overloaded by checking their calendars from the user tree.


Associate some tasks to a sprint

A sprint will contain some tasks. To associate some tasks to one sprint:
  • in the tree, select the sprint
  • on the right panel, select the Backlog (this is the terminology in Scrum) tab
  • un-toggle the select filter button to display the complete task tree
  • check all the tasks that need to be associated to the current sprint
  • (opt.) re-toggle the select filter button to display only the selected tasks
  • click on submit




Daily update progress of the tasks of a sprint

On a daily basis, you can update the progress of the tasks in a sprint:
  • in the tree, select the sprint
  • on the right panel, select the Backlog tab
  • move the progress bar of each task. While you’re moving the sliders:
    • the percentage of progress is update in the next column
    • the equivalent in days is updated in the next column
    • the last column indicates the estimated number of days for this task
  • click on submit

See the velocity charts/check the status of the sprint

  • in the tree, select the sprint
  • on the right panel, select the Velocity tab
  • some graph are displayed showing several useful information:

The top graph shows:
  • the theoretical curve of progress (the dashed red line)
  • the cumulated breakdown of all the tasks in this sprint

The bottom graph shows:
  • the theoretical curve of progress (the dashed red line)
  • the two curves representing the deal (top blue line) and real (bottom blue line) curves representing what we can do theoretically with the resources allocated
  • the status of the project:
    • Red area means that the project is at risk
    • Green means that the project is in a good shape

Test Tree

Here is a typical test tree. It immediately shows a number of useful information:
  • the total number of tests and testcases
  • the number of tests and testcases in each category
  • the number of tests and testcases in each folder


This screen is a bit different from the others since the left panel includes 2 separate areas:
  • the usual tree including all the tests that we have on the system
  • a sub-tree on the bottom that will list the testcases corresponding to the currently selected test
Tests can be arranged by placing them in specific folders. For instance, the tests can be organized the same way as the specifications. But if you’re testing a software API, it’s a common way of doing to have one test per function/method and group these functions by groups. You can then extend your test suite by adding stress test or negative tests etc. so probably need additional folders.

Tests and testcases are obviously what is the most important in the Data Model definition. The normal process is to start from specifications and make as many tests and testcases as necessary to check that specifications are all properly functioning.


Create a test

To create a test:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create test button
  • a dialog box including three tabs is displayed
  • fill the Details tab with the name and priority of the test (leave the canonical path blank).
  • fill the Testplan tab with the prerequisites and the general description of the test. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted.
  • Pick one user in the Author tab that will be registered as the author or the test
  • click on submit
  • immediately, the test appears in tree



Associate a test to specifications

A test is probably coming from at least one specification. To associate a test to one or several specifications:
  • in the tree, select the test
  • on the right panel, select the Specifications tab
  • un-toggle the select select filter button to display the complete specification tree
  • check all the specification that need to be associated to the current test
  • (opt.) re-toggle the select filter button to display only the selected specifications
  • click on submit



Create a testcase

We already mentioned that, for instance, a test could verify a specific function of an API. But there are a lot of things to check to validate one single function. You may want to test all combination of parameters and check that the result is correct.


One testcase must be able to check one specific function in some particular conditions. The sum of all the testcases makes one test. To have a test succeeding, all the testcase must succeed.


To create a testcase:
  • in the tree, select a test
  • on the right panel, click on the create testcase button
  • a dialog box including two tabs is displayed





  • fill the Details tab with the index (defining the order in which the testcases will be executed within a test), name and general description of the testcase. You can use the formatting tools (wiki-style) to format the text. Later, in reports this text will appear correctly formatted. Do not forget to check the Implemented checkbox. Non-implemented test are NOT executed when running a campaign. In case of manual test, always check the Implemented checkbox
  • select the Testplan tab and define all the steps and checks that will be needed in this testcase:
    • add a step:
      • in the tree, select the root folder
      • click on the create step button
      • enter the description of the step and submit
    • add parameters (opt.):
      • in the tree, select the parameters node
      • click on the create parameters button
      • enter the description of the parameter and submit
      • repeat the operation if you need to specify more parameters
    • add checks (opt.):
      • in the tree, select the checks node
      • click on one of the create boolean operators button
      • click on the new operator and click on the create check button
      • enter the description of the check and submit
      • repeat the operation if you need to specify more checks (you can mix as many different boolean operators as you want)
    • repeat the operation for every step and submit
  • click on submit
  • immediately, the testcase appears in the sub-tree on the left panel



Mofify the procedure of a test case

To modify the test procedure of a test case:
  • in the tree, select a test
  • in the sub-tree, select a testcase
  • on the right panel (Testplan tab), select the node in the tree you wish to modify
  • click on the edit a node button
  • apply the changes and submit




Campaign Tree

Here is a typical campaign tree. It immediately shows a number of useful information:
  • the total number of campaigns and campaign sessions
  • the number of campaigns and campaign sessions in each category
  • the number of campaigns and campaign sessions in each folder
  • the status (stopped, paused, running, idle) of each campaign session (indicated by the overlay on the icon)
  • the start and stop date and time of each campaign session




Once all the tests have been defined and implemented, you have to run them. To do so, you’ll first have to gather all the tests you which to run in a campaign. For instance, it may be of interest to define a campaign including all the tests of a specific category (or just a subset). A campaign is by definition just an ordered list of tests.

At this stage, you would be able to run a campaign but what happens if you run a campaign several times? of course you want to be able to retrieve results from each runs of a campaign. Here comes the Campaign Session.

You can create from a campaign as many campaign sessions as you want. This allows to archive independently all the results and report of each execution of the tests.


Create a campaign

To create a campaign:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create campaign icon
  • a dialog box including two tabs is displayed
    •  
  • fill the Details tab with the name of the campaign
  • in the Content tab, select all the tests you want to be part of this campaign
  • click on submit
  • immediately, the campaign appears in the tree

Order the tests in the campaign

  • in the tree, select the campaign
  • on the right panel, select the Order tab
  • select one or several tests in the list and use the move buttons to position the test(s) wherever you want in the list
  • click on submit

Create a campaign session

To create a campaign session:
  • in the tree, select a campaign
  • on the right panel, click on the create campaign session button
  • a dialog box including seven tabs is displayed
  • fill the Details tab with the name of the session.
  • opt.) select the Test operator tab and pick the user who will run the session
  • select the Agent tab and pick the agent on which the session will be run. If you wish to run the tests on the local host, leave the default selection (that should already match the local host)
  • select the SUT tab and pick the SUT on which the session will be run
  • select the Configuration tab and pick the configuration you which for each category involved in this campaign session. Once a campaign session is created, it is impossible (by purpose) to change the configurations associated. If no configuration are available:
    • click on the create configuration button
    • a dialog box is displayed
    • enter the name of the configuration
    • fill in all the forms displayed and submit
    • the launchers (xml and jar files) needed for this campaign session MUST be accessible in the bin/launchers folder
  • (opt.) if needed, modify the dynamic attributes values in the Attributes tab
  • (opt.) select the CC Emails tab and check some users to receive a notification when the campaign session will be completed
  • click on submit
  • immediately, the campaign session appears in the tree in idle state

Run a campaign session

To run a campaign session:
  • in the tree, select a campaign session
  • on the right panel, click on the start button
  • immediately, the Test campaign details screen appears and will display real time the results of testing
  • If the campaign includes tests to be executed manually (the tests are part of one or several categories where you choose manual.jar or simple_manual.jar as launcher), then you will get additional dialog boxes such as:

See the execution details

To see the details of the campaign session execution:
  • in the tree, select a campaign session
  • on the right panel, select the Content tab



  • the screen is split in three different areas
    • a test tree showing the tests included in the campaign with their results. It shows a number of useful information:
      • the total number of tests that succeeded or failed
        success
        failed
        unknown
      • the number of tests that succeeded or failed in each category
      • the number of tests that succeeded or failed in each folder
    • a testcase sub-tree showing the result of each testcase
      success
      failed
      not executed
      relative (some tests – i.e. performances - may just return some values that will need to be analyzed by an operator)
    • a message area showing the details of execution of one specific testcase
The process to get all the details of an execution is the following:
  • click on a test
  • the testcase sub-tree is updated showing the associated testcases results
  • click on a testcase
  • the message area show the details of each step during the testcase execution

Get the results

To see the results of the campaign session execution:
  • in the tree, select a campaign session
  • on the right panel, select the Results tab



The screen is split in two columns:
  • the Tests column gives the statistics/results only based on the tests results
  • the Testcases column gives the statistics/results only based on the testcases results
Each column immediately shows a number of useful information:
  • in the header tables:
    • the percentage of success, failure etc. of tests/testcases
    • the coverage of the execution (based on the number of tests/testcases that were not executed)
  • in the pie charts:
    • the number of success, failure etc. of tests/testcases
    • the percentage of success and failure (after removal of the unknown, relative or not executed tests/testcases)

Check the progression of a campaign

A Statistics tab is present on any campaign. This tab shows the progression/regression of this campaign. The information is taken from all the sessions executed belonging to the campaign. All the results are put in a graph so that the evolution (progression or regression) in time is clearly visible. The last session's results are also displayed. The new tab is split in 2 tabbed panes: one that shows the evolution of tests results and another one going deeper at the testcase level:




Defect Tree

Here is a typical defect tree. It immediately shows a number of useful information:
  • the total number of defects
  • the number of defects in each category
  • the number of defects in each folder
  • the status of each defect (indicated by the color of the icon)
  • the severity of each specification (indicated by the column)
  • the priority of each specification (indicated by the column)





Executing some tests (running some campaign session) makes you able to generate reports. It’s good to have a static view of what is working and what is not but it would be even better to link the tests that failed to actual defects. Hence, several failing testcases may be due to only one single defect.

At this point, a report analysis must be done by the test operator. This analysis should lead to associate all failed testcases to some defects.


Create a defect

To create a defect:
  • in the tree, select a folder (create one if necessary)
  • on the right panel, click on the create defect button
  • a dialog box including three tabs is displayed





  • fill the Details tab with the name, description, steps to reproduce, reproducibility, platform, operating system, status, severity and priority
  • pick one user in the Assigned to tab who will be registered as the one assigned to resolve the defect
  • in the Found in tab, check all the SUTs on which this defect can be observed
  • click on submit
  • immediately, the defect appears in the tree

Edit a defect

The defect will probably be edited several times by different people and will go through a complete workflow including five states:

Overlay Represents
New New defect
Assigned The defect has been assigned to a person to resolve it
Ack The defect is being investigated
Resolved The defect has been declared as resolved
Closed The fix has been verified and the defect has been closed



To edit a defect:
  • In the tree, select a defect
  • on the right panel (details tab), edit the information you wish and submit
Some fields will be accessible only when a certain status is reached:

Assigned:
Correction target date

Ack:
Correction target version
Completion %

Resolved/Closed:
Correction type
Correction description
Correction patches

Note: For better tracking purposes, when setting a defect’s status to Resolved, you should also pick one SUT in the Fixed in tab.


Link a failed test to a defect

To link a failed test to a defect:
  • switch to the Campaigns tab on the left panel
  • in the tree, select a campaign
  • on the right panel, select the Content tab
  • select a test with status failed or unknown
  • click on the Link to a defect button
  • a dialog box is displayed
  • un-toggle the select filter button to display the complete defect tree
  • check all the defects that need to be associated to the test execution
  • (opt.) re-toggle the select filter button to display only the selected requirement
  • click on submit