Package org.evoludo.simulator.exec

Class TestEvoLudo

Object
TestEvoLudo
All Implemented Interfaces:
MilestoneListener
public class TestEvoLudo extends Object implements MilestoneListener
TestEvoLudo is a test suite for EvoLudo. It generates test cases from a generator file or directory and compares the results with reference files. The test suite can be run in two modes: (1) generating test cases from a generator file or directory, and (2) testing the generated or existing test cases. The test suite accepts the following command line options:
  • --generate <filename>: read option sets from file and generate test cases
  • --tests <directory>: directory for storing/retrieving test cases (defaults to references)
  • --references <directory>: directory for storing/retrieving test cases
  • --reports <directory>: directory for storing failed test reports
  • --compress: compress generated test files
  • --minor: dump differences for minor failures
  • --verb: verbose mode
  • --help, -h or no arguments: this help screen
Author:
Christoph Hauert

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    (package private) static enum 
    TestEvoLudo.ConsoleColors
    The control codes for changing the style of the console output.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    (package private) boolean
    dumpMinor
    The flag to indicate whether to dump differences for minor failures.
    (package private) EvoLudoJRE
    engine
    Pointer to engine.
    (package private) File
    generator
    Directory with generator scripts for tests.
    (package private) boolean
    isRunning
    The flag to indicate whether tests are running.
    (package private) Logger
    logger
    Logger for keeping track of and reporting events and issues.
    (package private) int
    nTestFailures
    The number of failed tests.
    (package private) int
    nTestMinor
    The number of tests failing with minor errors.
    (package private) int
    nTests
    The total number of tests.
    (package private) int
    nTestWarnings
    The number of tests with warnings.
    (package private) boolean
    performTest
    The flag to indicate whether to perform tests or generate test cases.
    (package private) File
    referencesDir
    Directory with reference results.
    (package private) File
    reportsDir
    Directory for storing reports of failed tests.
    (package private) File
    testsDir
    Directory containing tests or for storing generated tests.
    (package private) boolean
    useCompression
    The flag to indicate whether to use compression for the generated test files.
    (package private) boolean
    verbose
    The flag to indicate whether to run in verbose mode.

  • Constructor Summary

    Constructors
    Constructor
    Description
    TestEvoLudo()
    Constructor for TestEvoLudo.

  • Method Summary

    Modifier and Type
    Method
    Description
    private File
    checkReference(File references, Plist result, String refname)
    Search for reference file refname in directory references.
    private boolean
    compareRuns(File refname, Plist reference, Plist replicate)
    Compare reference output reference with the output of the test replicate and generate a report if differences are found.
    void
    generate()
    Generate test cases from generator file or directory.
    protected void
    generate(File clo)
    Generate test cases from generator file clo.
    private String
    generateExportFilename(String clo, int idx)
    Generate export filename from command line options clo and index idx.
    void
    help()
    Print help screen.
    void
    logError(String msg)
    Log error message in red color to console or logger.
    void
    logMessage(String msg)
    Log message to console or logger.
    void
    logOk(String msg)
    Log message in green color to console or logger.
    void
    logTitle(String msg)
    Log bold message to console or logger.
    void
    logWarning(String msg)
    Log warning message in yellow color to console or logger.
    static void
    main(String[] args)
    Entry point for test routines of EvoLudo.
    void
    modelStopped()
    Called after a running EvoLudo model stopped because the model converged (or reached an absorbing state).
    void
    parse(String[] args)
    Parse the command line options.
    boolean
    run()
    Generate or run all tests.
    private boolean
    runModule(String task, String clo)
    Load and run module with parameters clo.
    private static File
    search(File file, String search)
    Recursively search for a file with name search in directory file.
    private String
    stripExport(String clo)
    Strip export option from command line options clo.
    void
    test(File dir)
    Test all files in directory dir.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface MilestoneListener

    modelDidInit, modelDidReset, modelLoaded, modelRelaxed, modelRunning, modelSettings, modelUnloaded, moduleLoaded, moduleRestored, moduleUnloaded

  • Field Details

    • engine

      EvoLudoJRE engine
      Pointer to engine. Engine has EvoLudoJRE class but do not rely on JRE specifics.

    • logger

      Logger logger
      Logger for keeping track of and reporting events and issues.

    • testsDir

      File testsDir
      Directory containing tests or for storing generated tests.

    • reportsDir

      File reportsDir
      Directory for storing reports of failed tests.

    • referencesDir

      File referencesDir
      Directory with reference results.

    • generator

      File generator
      Directory with generator scripts for tests.

    • performTest

      boolean performTest
      The flag to indicate whether to perform tests or generate test cases.

    • isRunning

      boolean isRunning
      The flag to indicate whether tests are running.

    • useCompression

      boolean useCompression
      The flag to indicate whether to use compression for the generated test files.

    • dumpMinor

      boolean dumpMinor
      The flag to indicate whether to dump differences for minor failures.

    • verbose

      boolean verbose
      The flag to indicate whether to run in verbose mode.

    • nTests

      int nTests
      The total number of tests.

    • nTestFailures

      int nTestFailures
      The number of failed tests.

    • nTestMinor

      int nTestMinor
      The number of tests failing with minor errors.

    • nTestWarnings

      int nTestWarnings
      The number of tests with warnings.

  • Constructor Details

    • TestEvoLudo

      public TestEvoLudo()
      Constructor for TestEvoLudo.

  • Method Details

    • generate

      public void generate()
      Generate test cases from generator file or directory.

    • generate

      protected void generate(File clo)
      Generate test cases from generator file clo.
      Parameters:
      clo - the generator file

    • generateExportFilename

      private String generateExportFilename(String clo, int idx)
      Generate export filename from command line options clo and index idx.
      Parameters:
      clo - the command line options
      idx - the index of the test
      Returns:
      the export filename

    • stripExport

      private String stripExport(String clo)
      Strip export option from command line options clo.
      Parameters:
      clo - the command line options
      Returns:
      the stripped command line options

    • runModule

      private boolean runModule(String task, String clo)
      Load and run module with parameters clo. The options --seed 0 is prepended and --delay 0 is appended to clo. Because for options that are specified multiple times the latter takes precedence, this ensures that the results are reproducible with potentially custom seeds and also run at full speed. Returns false if the module didn't run and hence no export was generated, even if requested with the --export option. This happens for example with --timestop 0.
      Parameters:
      task - the task that is running (generating or testing)
      clo - the command line options for running the module
      Returns:
      true if the module ran successfully

    • compareRuns

      private boolean compareRuns(File refname, Plist reference, Plist replicate)
      Compare reference output reference with the output of the test replicate and generate a report if differences are found. The reference is stored in refname. The method returns true if the test passed.
      Parameters:
      refname - the name of the reference file
      reference - the reference Plist
      replicate - the replicate Plist
      Returns:
      true if the test passed

    • test

      public void test(File dir)
      Test all files in directory dir. This directory can either contain test files or files for generating them. In either case the test output is compared to the reference files.
      Parameters:
      dir - the directory with test files

    • checkReference

      private File checkReference(File references, Plist result, String refname)
      Search for reference file refname in directory references. If refname is found verify the results in the plist result.
      Parameters:
      references - the directory with reference files
      result - the Plist with the results
      refname - the name of the reference file (if it exists)
      Returns:
      if verification successful return File pointing to reference; if unsuccesssful return null; and if reference file not found return references

    • search

      private static File search(File file, String search)
      Recursively search for a file with name search in directory file.
      Parameters:
      file - the directory to search
      search - the name of the file to search for
      Returns:
      the file if found, otherwise null

    • modelStopped

      public void modelStopped()
      Description copied from interface: MilestoneListener
      Called after a running EvoLudo model stopped because the model converged (or reached an absorbing state).
      Specified by:
      modelStopped in interface MilestoneListener

    • run

      public boolean run()
      Generate or run all tests.
      Returns:
      true if successful

    • parse

      public void parse(String[] args)
      Parse the command line options.
      Parameters:
      args - the command line options

    • help

      public void help()
      Print help screen.

    • main

      public static void main(String[] args)
      Entry point for test routines of EvoLudo.
      Parameters:
      args - the string of command line arguments

    • logMessage

      public void logMessage(String msg)
      Log message to console or logger.
      Parameters:
      msg - the message to log

    • logTitle

      public void logTitle(String msg)
      Log bold message to console or logger.
      Parameters:
      msg - the message to log

    • logOk

      public void logOk(String msg)
      Log message in green color to console or logger.
      Parameters:
      msg - the message to log

    • logWarning

      public void logWarning(String msg)
      Log warning message in yellow color to console or logger.
      Parameters:
      msg - the warning to log

    • logError

      public void logError(String msg)
      Log error message in red color to console or logger.
      Parameters:
      msg - the error to log