4 CVRANAL - Coverage Analysis

4.1 Introduction

CVRANAL reads the coverage (.cvr) files written by the test version of a program and writes a report showing which code blocks have never been executed in the current series of tests, and which have been executed most frequently (execution hot-spots). Based on this report, it should be possible to devise additional tests to reduce the amount of untested code. The analysis of execution hot-spots may also suggest ways to make the program run faster.

CVRANAL is also able to merge the execution frequency data with the original source code (or a copy thereof). The number of times each block has been executed is inserted as a comment, usually in columns 73-80, and preceded by a '!' character.

Coverage Analysis is an important tool for anyone involved in software quality assurance. Clearly a program cannot be said to have been properly tested if there are parts of the source code which have never been executed, and programmers are often surprised to find that even quite extensive testing can leave large amounts of untested code. Any thorough QA procedure should use coverage analysis in conjunction with both static and dynamic analysis, since each approach will find problems that the others cannot detect.

4.2 The CVRANAL Command Line

To start a run, simply type cvranal followed by the name of the coverage file(s) you wish to process. You can use wild-cards to specify more than one file. For example:

      cvranal *.cvr

processes the contents of all .cvr files in the current directory.

Under DOS/Windows and VMS, .cvr is assumed if no extension is specified (b* is treated as b*.cvr). The default file name is *.cvr. Thus, if you just type cvranal, all .cvr files in the current directory are processed.

Under UNIX, the extension must be specified, and there is no default file name.

The operation of CVRANAL may be modified by command line switches. Currently the following switches are available:

For example

      cvranal a*.cvr to=cvrfred.out fig=cvrfred.fig  513=0

This command causes CVRANAL to analyze all files in the current directory whose name fits the template a*.cvr. Configuration options are read from cvrfred.fig, but with item 513 reset to 0. The output report is sent to cvrfred.out.

4.3 The CVRANAL Configuration File

The CVRANAL configuration file has the same general format as the SPAG and GXCHK configuration files (see Section 2.5, 3.3). The item numbers for CVRANAL are in the range 500-600, and all except one are integers. There is no overlap between CVRANAL, GXCHK and SPAG item numbers.

An example configuration file is shown below. The meaning of each data item is explained in detail in the next section. The values shown in this example are, in fact, the defaults which are used by CVRANAL if no configuration file is found.

Example CVRANAL Configuration File

 =========================== CVRANAL controls ==============================

501=0   0 - normal run                        1 - set usage counts to zero
502=0   0 - system default box drawing chars  1 - use IBM PC box drawing chars
        2 - use only ASCII characters
503=200 number of lines/page in CVRANAL.OUT (if >=200 no pages within reports)
504=78  width of output reports

510=2   0=No check for unexercised code
        1=Report unexercised code to CVRANAL.OUT
        2=line number summary to CVRANAL.OUT
511=200 Number of hotspots to CVRANAL.OUT
512=7   Number of hotspots to screen
513=73  0=don't modify source code  >0  insert usage count at column n

550=          List (separated by ;) of directories containing source code

4.4 CVRANAL Configuration Data

4.5 Using CVRANAL

4.5.1 Source Code Changes

CVRANAL may be used as often as required, at any stage during the testing process. However if the source code is changed in any way, it is necessary to use SPAG to re-instrument the changed code before compiling and linking the revised test program. Once this is done, we recommend that you re-initialize all the coverage data (by running CVRANAL with item 501 set to 1), even for source files which have not been changed. If this is not done, the significance of the execution hot-spot data is questionable, as different parts of the code will be using different base points.

4.5.2 Execution Count Overflow

In the probe routines as supplied, execution counts are stored in a standard Fortran INTEGER array, and there is no guard against overflow. On most machines, this means that if a block of code is executed more than about 2147 million times, the execution count becomes very large and negative. In most cases, this is an acceptable compromize, as it is rare for execution counts to be so large. However, a version of the probe routines in which execution counts are double precision floating point numbers may easily be constructed by changing the type of the variable TCOunt from INTEGER to DOUBLE PRECISION in the probes routine include file (probes.prv in the plusFORT installation directory). This modified version is slower, and uses more memory, but has an effectively unlimited capacity.