Test Suite News: 4/08
We're in the process of moving to a new combined test suite, which will deprecate the information below. You can get a copy of the new test suite through anonymous svn checkout:
svn co http://code.roadsend.com/svn/pcc/trunk/pcc-testsuite
It should be used after pcc has already been created and installed. Simply type "make" to run the tests.
Stand Alone Test Suite
A small test suite is included to verify pcc has compiled correctly and is in working order. To run it, type "make test" from the source tree after building with "make"
Zend Compatibility Test Suite
pcc uses the "dotest" program (located in the source tree root directory) to conduct functionality and regression tests.
Prerequisite
Because the test program compares the output of compiled scripts to the output of the same scripts when run through Zend PHP (to ensure compatibility), Zend PHP is required and should be available in the system PATH. Be sure to either copy the default config file (location in {root}/doc/pcc.conf) to /etc/pcc.conf, or else set the PCC_CONF environment variable to the proper location before running the suite.
General Usage
dotest <test scripts dir> <output dir>
The dotest program takes two arguments: a directory containing a list of input scripts, and an output directory where the build and output files will be written.
dotest will get a list of all scripts in the input directory and then:
- compile the script
- run and time the compiled script, saving the output
- (optionally) run and time the script interpreted, saving the output
- run and time the script with Zend PHP, saving the output
- compare the output from the compiled (and possibly interpreted script) to the Zend PHP output
- the tests PASSES when the output matches the output from the Zend run
- display timing information
Test Run Configuration
Several environment variables affect the functionality of dotest:
- INTERPRET
- set this to 1 to test the scripts interpreted as well as compiled
- OPTIMIZE
- set this to 1 to run the tests in "production" (instead of "debug") mode
- TEST
- name of single script to run from the input directory, instead of all scripts. if this is set, only the specified test will be run, and if the test fails, a diff of the output will automatically be shown upon completion.
- TEST_CYCLES
- the number of times to run the script, for timing purposes (default 1). when benchmarking, results will be more accurate when this number is 10 or greater.
- PCC_HOME
- the compiler source tree root directory
- PCC_CONF
- full path to the PCC config file. default location is /etc/pcc.conf -- either copy the default ({root}/doc/pcc.conf) there, or set the PCC_CONF variable to the correct location.
- PCC_OPTS
- additional arguments to pass on the command line to pcc
- PHP
- full path of Zend PHP binary to use
Known Failures
The currently list of known failures, and their reason for failure, is maintained here.
Running Test Suites
Roadsend Compiler comes with several test suites:
Main Test Suite
This suite tests the basic functionality of the compiler and runtime. The tests are located in the {root}/tests/ directory. To run the suite, simply run "make check" in the source tree {root} directory. All tests should pass - with the exception of a few that fail because of output differences of floating point numbers between PCC and Zend PHP.
Regression Test Suite
This suite attempts to verify that bugs which have already been fixed have not regressed. The tests are located in the {root}/bugs/tests/ directory. To run the suite, type "make check" in the {root}/bugs/ directory. There are a few known failures on various platforms; they are noted in the "known-failures" file.
Extension Test Suites
These tests are meant to verify the functionality of the various extensions (located in {root}/runtime/ext/). Each extension directory has a "tests" directory. To run the test suite, type "make check" in the appropriate extension's directory.
Benchmark Suite
These tests are meant to test the speed of the compiler. The tests are located in the {root}/benchmarks/tests/ directory. To run the benchmark test suite, type "make check" in the {root}/benchmarks/ directory.
Zend Suite
These are tests from the Zend PHP distribution. Currently these are only run through the interpreter.
