Launchers / Automated tests

Proxy Launcher (proxy.jar)

One big issue in testing on distributed environments is how to "publish" scripts execution simultaneously on different components.

The Proxy launcher allows spliting a test in several scripts and executing them simultaneously on different hosts.

Let's imagine you have to test a VoIP framework that comprises 3 different machines: a Client, a Gateway and a Server. To test the data flow, you will need to deploy for each test (at least) 3 scripts on each of those and run them at the same time.

The Proxy launcher is used in conjunction with a special kind of agent called the sub-agent (XSubAgent in the distribution package). A sub-agent is a very basic agent embedding an XML-RPC server and which communicates only through this media with the main agent.

proxy


or
proxy


Here is the flow in the above example:
  • The test campaign in XStudio contains only one test Test1
  • The test belongs to a category associated with the proxy.jar launcher
  • A campaign session is created to execute the campaign on a specific XAgent and with a specific configuration for the proxy launcher.
  • This configuration includes 3 sub-agents. For each of those sub-agents, the sub-configuration include:
    • the sub-agent's XML-RPC Url
    • a launcher (launcher1.jar, launcher2.jar and launcher3.jar)
    • a configuration
    • and a script postfix (_A, _B and _C)
  • The proxy launcher contacts individually and simultaneously all the sub-agents and forward the requests to execute sub-tests Test1_A, Test1_B and Test1_C
  • Each sub-agent will execute its specific script and will return results to the proxy launcher. Each script can have the capability to interact/synchronize with the others as they are running simultaneously.
  • The proxy launcher consolidates all the results and logs and returns it to XStudio.

Hence, one single test is split by the proxy launcher in different scripts that are executed simultaneously on different XSubAgents located on the Client, the Gateway and the Server of our VoIP Framework.

Configuration

The proxy.xml file is just a template and must NOT be edited. It's used by the system to build dynamically the form that the user will be able to fill in from the GUI when creating a custom execution configuration.

Parameter Description
SubAgent
XML-RPC URL This must indicate the url of the SubAgent.

Default value is: http://localhost:8282
Launcher This must indicates which launcher the sub-agent will need to use to execute its script.
Default value is: random.jar
Configuration This must indicate which configuration the sub-agent will need to use to execute its script.

Default value is: default
Script postfix This must indicate the postfix used to determine the name of the script to run by the sub-agent.
If the test name is Test1 and we configure this parameter to _agent1.py, then the script that the sub-agent's launcher will get and execute is Test1_agent1.py
Default value is: _agent1

These values can be changed while creating the campaign session from XStudio.
Note about file path parameters:
Any parameter referring to a file or folder path (for instance Test root path) can be provided either using \ separator (if the tests are going to be executed on a Windows agent) or / separator (if the tests are going to be executed on a linux or MacOSX agent).

On windows, if you provide a path containing an OS-localizable folder such as C:\Program Files, always prefer the English version (i.e. NOT C:\Programmes if you're using a french-localized Windows) or the corresponding native environment variable (i.e. %PROGRAMFILES%).


XSubAgent

XSubAgent is included in the XAgent package.
XSubAgent embeds an XML-RPX server and is listening by default on port 8282 but you can specify another port by adding to the command line --port <port>

Limitations

  • Conversely to the standard XAgent, there is no mecanism to handle several sessions on the same XSubAgent (pooling). If you run several simultaneous sessions dispatching their tests to a common XSubAgent, it may crash.
  • If one of the XSubAgent is not properly running at runtime, An Exception will be thrown and the session is frozen



Permissions

WARNING: if you're running your tests on Windows, it may be required to run the tests as administrator.
Having an account with Administrators permissions may even not be enough in some cases (especially if you're using Windows 10) and you may need to disable completely the UAC (User Access Control) on your computer.
To do so:
  • Press the Windows + R key combination
  • Type in regedit
  • Go to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System
  • In the right-side pane, look for EnableLUA and set the value 0
  • Close the registry editor
  • Restart your computer

Debug

If your tests are not executed correctly or are reporting only failures, this is very likely because your configuration is incorrect or because you used a wrong naming convention for your tests and test cases.

The best way to quickly find out what's wrong is to look at the traces generated by XStudio (or XAgent).
The traces always include the detailed description of what the launcher performs (command line execution, script execution, API calling etc.) to run a test case. So, if you experiment some problems, the first thing to do is to activate the traces and look at what's happening when you run your tests.

Then, try to execute manually in a cmd box the exact same commands.
This will normally fail the same way.
At this point, you needs to figure out what has to be changed in these commands in order to have them run properly.

When you have something working, compare these commands to what's described in the Process chapter above. This will tell you exactly what you need to change.

Most of the time, this is related to:
  • some incorrect values in some parameters of your configuration,
  • the name of your tests,
  • the name of your test cases,
  • the canonical path of your tests