Package org.evoludo.simulator.exec
Class TestEvoLudo
Object
TestEvoLudo
- All Implemented Interfaces:
MilestoneListener
TestEvoLudo is a test suite for EvoLudo. It generates test cases from a
generator file or directory and compares the results with reference files.
The test suite can be run in two modes: (1) generating test cases from a
generator file or directory, and (2) testing the generated or existing test
cases. The test suite accepts the following command line options:
--generate <filename>
: read option sets from file and generate test cases--tests <directory>
: directory for storing/retrieving test cases (defaults to references)--references <directory>
: directory for storing/retrieving test cases--reports <directory>
: directory for storing failed test reports--compress
: compress generated test files--minor
: dump differences for minor failures--verb
: verbose mode--help
,-h
or no arguments: this help screen
- Author:
- Christoph Hauert
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescription(package private) static enum
The control codes for changing the style of the console output. -
Field Summary
FieldsModifier and TypeFieldDescription(package private) boolean
The flag to indicate whether to dump differences for minor failures.(package private) EvoLudoJRE
Pointer to engine.(package private) File
Directory with generator scripts for tests.(package private) boolean
The flag to indicate whether tests are running.(package private) Logger
Logger for keeping track of and reporting events and issues.(package private) int
The number of failed tests.(package private) int
The number of tests failing with minor errors.(package private) int
The total number of tests.(package private) int
The number of tests with warnings.(package private) boolean
The flag to indicate whether to perform tests or generate test cases.(package private) File
Directory with reference results.(package private) File
Directory for storing reports of failed tests.(package private) File
Directory containing tests or for storing generated tests.(package private) boolean
The flag to indicate whether to use compression for the generated test files.(package private) boolean
The flag to indicate whether to run in verbose mode. -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprivate File
checkReference
(File references, Plist result, String refname) Search for reference filerefname
in directoryreferences
.private boolean
compareRuns
(File refname, Plist reference, Plist replicate) Compare reference outputreference
with the output of the testreplicate
and generate a report if differences are found.void
generate()
Generate test cases from generator file or directory.protected void
Generate test cases from generator fileclo
.private String
generateExportFilename
(String clo, int idx) Generate export filename from command line optionsclo
and indexidx
.void
help()
Print help screen.void
Log error message in red color to console or logger.void
logMessage
(String msg) Log message to console or logger.void
Log message in green color to console or logger.void
Log bold message to console or logger.void
logWarning
(String msg) Log warning message in yellow color to console or logger.static void
Entry point for test routines of EvoLudo.void
Called after a running EvoLudo model stopped because the model converged (or reached an absorbing state).void
Parse the command line options.boolean
run()
Generate or run all tests.private boolean
Load and run module with parametersclo
.private static File
Recursively search for a file with namesearch
in directoryfile
.private String
stripExport
(String clo) Strip export option from command line optionsclo
.void
Test all files in directorydir
.Methods inherited from class Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface MilestoneListener
modelDidInit, modelDidReset, modelLoaded, modelRelaxed, modelRunning, modelSettings, modelUnloaded, moduleLoaded, moduleRestored, moduleUnloaded
-
Field Details
-
engine
EvoLudoJRE enginePointer to engine. Engine has EvoLudoJRE class but do not rely on JRE specifics. -
logger
Logger loggerLogger for keeping track of and reporting events and issues. -
testsDir
File testsDirDirectory containing tests or for storing generated tests. -
reportsDir
File reportsDirDirectory for storing reports of failed tests. -
referencesDir
File referencesDirDirectory with reference results. -
generator
File generatorDirectory with generator scripts for tests. -
performTest
boolean performTestThe flag to indicate whether to perform tests or generate test cases. -
isRunning
boolean isRunningThe flag to indicate whether tests are running. -
useCompression
boolean useCompressionThe flag to indicate whether to use compression for the generated test files. -
dumpMinor
boolean dumpMinorThe flag to indicate whether to dump differences for minor failures. -
verbose
boolean verboseThe flag to indicate whether to run in verbose mode. -
nTests
int nTestsThe total number of tests. -
nTestFailures
int nTestFailuresThe number of failed tests. -
nTestMinor
int nTestMinorThe number of tests failing with minor errors. -
nTestWarnings
int nTestWarningsThe number of tests with warnings.
-
-
Constructor Details
-
TestEvoLudo
public TestEvoLudo()Constructor for TestEvoLudo.
-
-
Method Details
-
generate
public void generate()Generate test cases from generator file or directory. -
generate
Generate test cases from generator fileclo
.- Parameters:
clo
- the generator file
-
generateExportFilename
Generate export filename from command line optionsclo
and indexidx
.- Parameters:
clo
- the command line optionsidx
- the index of the test- Returns:
- the export filename
-
stripExport
Strip export option from command line optionsclo
.- Parameters:
clo
- the command line options- Returns:
- the stripped command line options
-
runModule
Load and run module with parametersclo
. The options--seed 0
is prepended and--delay 0
is appended toclo
. Because for options that are specified multiple times the latter takes precedence, this ensures that the results are reproducible with potentially custom seeds and also run at full speed. Returnsfalse
if the module didn't run and hence no export was generated, even if requested with the--export
option. This happens for example with--timestop 0
.- Parameters:
task
- the task that is running (generating or testing)clo
- the command line options for running the module- Returns:
true
if the module ran successfully
-
compareRuns
Compare reference outputreference
with the output of the testreplicate
and generate a report if differences are found. The reference is stored inrefname
. The method returnstrue
if the test passed.- Parameters:
refname
- the name of the reference filereference
- the referencePlist
replicate
- the replicatePlist
- Returns:
true
if the test passed
-
test
Test all files in directorydir
. This directory can either contain test files or files for generating them. In either case the test output is compared to the reference files.- Parameters:
dir
- the directory with test files
-
checkReference
Search for reference filerefname
in directoryreferences
. Ifrefname
is found verify the results in the plistresult
.- Parameters:
references
- the directory with reference filesresult
- thePlist
with the resultsrefname
- the name of the reference file (if it exists)- Returns:
- if verification successful return
File
pointing to reference; if unsuccesssful returnnull
; and if reference file not found returnreferences
-
search
Recursively search for a file with namesearch
in directoryfile
.- Parameters:
file
- the directory to searchsearch
- the name of the file to search for- Returns:
- the file if found, otherwise
null
-
modelStopped
public void modelStopped()Description copied from interface:MilestoneListener
Called after a running EvoLudo model stopped because the model converged (or reached an absorbing state).- Specified by:
modelStopped
in interfaceMilestoneListener
-
run
public boolean run()Generate or run all tests.- Returns:
true
if successful
-
parse
Parse the command line options.- Parameters:
args
- the command line options
-
help
public void help()Print help screen. -
main
Entry point for test routines of EvoLudo.- Parameters:
args
- the string of command line arguments
-
logMessage
Log message to console or logger.- Parameters:
msg
- the message to log
-
logTitle
Log bold message to console or logger.- Parameters:
msg
- the message to log
-
logOk
Log message in green color to console or logger.- Parameters:
msg
- the message to log
-
logWarning
Log warning message in yellow color to console or logger.- Parameters:
msg
- the warning to log
-
logError
Log error message in red color to console or logger.- Parameters:
msg
- the error to log
-