Integration with ContinuousIntegration systems

XStudio fat client comes with a tool called XContinuousIntegration.
  • On Windows platforms, you can find it in the xstudio folder. The executable is named xcontinuousintegration_console.exe
  • On MacOSX and linux platforms, the executable script is named xcontinuousintegration.sh

Goal

XContinuousIntegration is aimed at being called by automatic build machines. Once a build has been run successfully, the build server can automatically trigger the creation and the start of a campaign session in XStudio. This way, after your overnight build has completed you can automatically run your automated test suites without having to attend.
This is even more convenient when we speak about continuous integration. In this case, each time a commit will occur on the versioning system, a build will be generated and the tests executed.
xcontinuousintegration integration

Usage

To trigger the creation and the execution of a campaign session, the build server will just have to call the XContinuousIntegration executable with the appropriate arguments which are...
Usage: xcontinuousintegration_console.exe --campaignId <campaignId>
                                          --agents <agentId>:<nbInstances>:<sync>,...
                                          --sutId <sutId>
                                          --configurations <categoryId>:<configId>,...
								  
Options:
	--sessionName               <sessionName>
	--operatorId                <userId>
	--agentsPick                <agentsPick>
	--ccEmailUserIds            <userId>,...
	--monitoringAgentId         <agentId>
	--monitoringConfId          <monitoringConfId>
	--ifParentFailed            <action>
	--ifParentUnknown           <action>
	--checkAgentOnline
	--synchronous
	--synchronousTimeout        <timeout>    (only with '--synchronous')         
	--generateReport                         (only with '--synchronous')              
	--reportTransform           <xsltName>   (only with '--synchronous' and '--generateReport')         
	--reportStyle               <styleName>  (only with '--synchronous' and '--generateReport')         
	--reportOutputPath          <path>       (only with '--synchronous' and '--generateReport')     
	--reportName                <reportName> (only with '--synchronous' and '--generateReport') 
	--reportIncludesAttachments    
	--overloadBooleanAttributes <testId>[<attributeId>:<attributeValue>;...],...
	--overloadIntegerAttributes <testId>[<attributeId>:<attributeValue>;...],...
	--overloadStringAttributes  <testId>[<attributeId>:<attributeValue>;...],...
	--overloadBooleanAttributes <paramId>:<paramValue>,...
	--overloadIntegerAttributes <paramId>:<paramValue>,...
	--overloadStringAttributes  <paramId>:<paramValue>,...

Where:
	<sync> can take the following values: 0=None,
                                              1=Synchronization on tests,
                                              2=Synchronization on test cases,
                                              3=In a row
												
	<agentsPick> can take the following values: 0=Any agent in the pool,
                                                    1=All agents in the pool (default value),
                                                    2=first available agent in the pool
												
	<action> can take the following values: 'EXECUTE', 
                                                'SET_AS_FAILED'
                                                'DO_NOT_EXECUTE'
																								
	<timeout> is a timeout given in seconds (if the session is not executed after this delay, 
	                XContinuousIntegration exits and no report is generated)
	<path> is a local path to an existing folder (i.e. 'C:/Users/eric')		
				

Where do I find all those Ids

As you see, this command line needs many ids. Hopefully you can get them pretty easily:

IdWhere to find it
Mandatory
campaignId From the campaign tree, select the campaign and check the header on the right panel
agentId From the agent tree, select the agent and check the header on the right panel
sutId From the SUT tree, select the sut and check the header on the right panel
categoryId From the test tree, select the category and check the header on the right panel
configurationId From the campaign tree, select a session that has already been executed using this configuration, select the Configuration tab and check for each category the id of the configuration
Optional
operatorId From the user tree, select the user and check the header on the right panel
userId From the user tree, select the user and check the header on the right panel
monitoringConfIdFrom the campaign tree, select a session that has already been executed using this monitoring configuration, select the Monitoring / Monitored features tab and check the id of the configuration
testId From the test tree, select the test and check the header on the right panel
attributeId Directly from the database in the table_attr_*
paramId Directly from the database in the table_param_*

IMPORTANT: all the ids must be provided WITHOUT their prefix!
For instance, if you have a campaign with the id C_1234, you just need to provide to XContinuousIntegration the value 1234.

Example

xcontinuousintegration_console.exe --campaignId 1090 --agents 16:1:0;17:2:2 --sutId 89
--configurations 80:42;81:13 --checkAgentOnline --synchronous --synchronousTimeout 14400
--generateReport --reportTransform HTML_Complete_with_stats --reportStyle XQual
--reportOutputPath E:/tmp --reportName my_last_report --reportIncludesAttachments
--overloadBooleanAttributes 123[12:0;22:1],134[12:1]

This command will run XContinuousIntegration in synchronous mode (it returns only when the session completed).

If at least one of the agents AG_16 and AG_17 are offline, the program exits immediately because of
the --checkAgentOnline argument). The program will return anyway if the session is not completed after 4 hours (14400 seconds).

When the session is completed, an HTML report named my_last_report.html is generated locally in E:/tmp based on the XSLT reportHTML_Complete_with_stats.xslt (presented in XStudio as (HTML) Complete_with_stats in the drop-down menu). A my_last_report.zip file is also created that contains all the resources (including attachments etc.) to have a fully indexed/emailable/attachable report.
In this session, the test T_123 will be overloaded with a boolean attribute 12 set to false and the attribute 22 set to true. The test T_134 will be overloaded with the attribute 12 set to true.

The linux version is also provided and use the same syntax except that the binary is a shell script. You can also run XContinuousIntegration by interpreting directly the jar file.
If you need to execute XContinuousIntegration remotely on a linux server for example, you can call it within an ssh session using a command similar to:
ssh -o StrictHostKeyChecking=no <account> "cd <path to xcontinuousintegration>;
java -jar xcontinuousintegration.jar <options as above>"

Of course you can also ensure you have the right certificates stored to avoid using the StrictHostKeyChecking=no option that is unsecure.
You may also need to download/upload some files from/to the remote server.
In this case, you can use scp a similar way.
For instance, to upload some files:
scp -o StrictHostKeyChecking=no * <account>:<destination path>
To download some files:
scp -o StrictHostKeyChecking=no <account>:<source path> .

Integration with third-party systems

XStudio can be integrated in ANY continuous integration environment as it just consists in executing a simple command after a build is completed.
However, the continuous integration system may even better integrate with XStudio if it can retrieve the results.

jenkins
Jenkins can also parse any test output that conforms to the standard JUnit XML format so you can use --reportTransform XML_JUnit as well with Jenkins.
Please refer to Jenkins documentation on how to include the XContinuousIntegration command line in the process.
Basically, in the build step click on Execute a shell script and type the command:
java -jar xcontinuousiontegration.jar with all the appropriate options.

To get some nice graphics and trend charts showing the evolution of your test results you can add a step in your build plan:
In Post-build Actions, click on Add post-build action and select Publish JUnit test result report. In the Test report XMLs field, specify the directory where all your xml reports are getting generated and you're done.

If you click on Test Result it will take you to the generated report. Here you will be able to see:
  • Test Results showing number of test failures out of whole tests
  • All Failed Tests list down the name of all the test methods which got failed
  • All Tests will show the duration of execution, number of test failed, number of tests skipped, number of test passed and total number of tests

bamboo
Bamboo (and others) can parse any test output that conforms to the standard JUnit XML format. You have just to specify the option --reportTransform XML_JUnit in the command line and a JUnit XML report will be additionally generated so that Bamboo can parse it and retrieve the results.
You can refer to Atlassion's tutorial on how to configure an external command in Bamboo.
Atlassian also have a page describing the JUnit XML format if you're interested.
hudson
Hudson can also parse any test output that conforms to the standard JUnit XML format so you can use --reportTransform XML_JUnit as well with Hudson.
Please refer to Hudson documentation on how to include the XContinuousIntegration command line in the process.
teamcity logo
TeamCity can also parse any test output that conforms to the standard JUnit XML format so you can use --reportTransform XML_JUnit as well with TeamCity.
Please refer to TeamCity documentation on how to include the XContinuousIntegration command line in the process.