eCAT - Continuous Agent Testing on Eclipse

I. Introduction
The specific features of agent systems (autonomicity, proactivity, learning ability, cooperation, etc.) demand for a framework that supports extensive and possibly automated testing.  eCAT was introduced to challenge multi-agent systems testing. The framework contains three main components: a special software agent (called Autonomous Tester Agent) plays the role of the human tester by producing test suites and executing them; Monitoring Agents observe the execution (including the exchanged messages), provide insights to the Autonomous Tester Agent, and recognize any behavior that does not comply with the agent specifications (given in the form of pre- and post-conditions); and a Test Suite Editor that generates initial test suites from agent specifications and design, and allows users to specify test input as needed. The architecture of eCAT is presented in the following figure. The operation of the framework is described as follows:
  1. Based on agent specifications and design (e.g., TAOM4E1 output), the Test Suite Editor can generate initial test suites and then provide GUI for
    end-users to edit them. They are considered as manual test suites.
  2. The Autonomous Tester Agent takes those test suites and/or generates other test suites randomly. It then continuously executes them against the
    multi-agent system under test. During the course of test execution, the Autonomous Tester Agent can evolve test suites by applying a mutation
    and evolutionary technique in order to create more test suites, which aim at revealing more bugs.
  3. The Monitoring Agent assists the Autonomous Tester Agent during testing. By monitoring events and communication acts happened in the multi-
    agent system and its environment, it provides useful information to the Autonomous Tester Agent in order to decide if a test successes of fails and
    the trace to the bug when failed.
The most important feature of eCAT is the capability of generating and evolving test cases continuously in the course of testing MAS in order to reveal faults. The MAS deployed in the testing environment is free to evolve according to the behavior of the composing agents, but eCAT monitors continuously the state of the other agents and automatically generate test data that are submitted to the MAS in order to guide their evolution towards revealing faults of the system.

eCAT architecture
eCAT consists of three main components: Test Suite Editor, that semi-automatically generates test suites skeleton and allows human testers to specify test data from goal analysis diagrams produced by  TAOM4E, a tool that supports Tropos;  Autonomous Tester Agent, a special  agent that is capable of executing test suites against a MAS; and Monitoring Agents, that monitor communications and events among agents to help debugging and diagnosing faults. See the figure on the right.
eCAT components

II. Screen shots (click on the figure to enlarge)
                          Test case generation eCAT execution                                
Test cases generation: eCAT takes a
Tropos / TAOM4E model as input. It then
analyzes and generates test suites skeletons
and assists user to define specific domain data

eCAT implementation consists of two parts, one
integrated in Eclipse IDE as plugin to facilitate developers
, the other runs in JADE to test software agents, taking into
account their nature.
eCAT - IDE test result
Execute test: it is easy to lauch test from the IDE Test result on one Node: eCAT provides results view, monitoring view and log view.
evol-mutation testing results evol-mutation testing results
Evolutionary-mutation testing: eCAT can run
test simultaneously on multiple nodes (instances of
JADE platform); testing results of the mutants of the MAS under test help evolving test suite in order to test the MAS itself.
Evolutionary-mutation testing 

III. Examples of test suites

Test Suite 1,
Test Suite 2, Test Suite 3, Test Configuration

III. Downloads
eCAT needs some more time to be documented, user who is interested in using it could contact cunduy [at] fbk [dot] eu!