Logging

The target definition file settings related to logging are as follows:

logfile            "/tmp/synthnand.log"
log                read write erase error
max_logfile_size   16M
number_of_logfiles 4
generate_checkpoint_images

The logfile setting controls the location of the logfile. The default is to add a suffix .log to the CYGDAT_NAND_SYNTH_FILENAME setting, thus creating logfiles in the same directory as the NAND image file.

The log setting specifies which events should be logged. It is followed by a list of some or all of the following: read, READ, write, WRITE, erase, and error. read logs all calls to the driver's page read function, but not the data actually read. READ is like read but logs the actual data as well as the event. Similarly write and WRITE log calls to the driver's page write function, without or with the data being written. erase logs calls to the driver's block erase function. error logs any bad block injection events.

Logging can quickly generate very large files, especially when READ or WRITE debugging is enabled. This can have unfortunate side effects, for example an overnight stress test can fail because the logfile has filled all available disk space. To avoid this it is possible to limit the size of each logfile using the max_logfile_size setting. This is simply a number followed by a unit K, M or G.

When max_logfile_size is exceeded the NAND driver takes appropriate action. The default behaviour is just to delete the current logfile and create a new one, with the same name as before. This can be unfortunate if some particularly interesting event happened just before the maximum logfile size was exceeded because all logging information related to that event will have been lost. To avoid this it is possible to have multiple logfiles, limited by the number_of_logfiles setting. Assume a logfile name of /tmp/synthnand.log, a maximum logfile size of 16M and four logfiles. After the first 16MB of logging data has been written to /tmp/synthnand.log that file will be renamed to /tmp/synthnand.log.0 and a new current logfile /tmp/synthnand.log will be created. After another 16MB the current logfile will be renamed to /tmp/synthnand.log.1, and so on. After 64MB the maximum allowed number of logfiles has been created, so /tmp/synthnand.log.0 will be deleted and then /tmp/synthnand.log will be renamed to /tmp/synthnand.log.3. Hence the maximum disk space used will be between 48MB and 64MB, plus a small amount of overflow for each logfile. All logfiles are in plain text, one event per line, and a single event will not be spread over multiple logfiles.

In addition to the logfiles the NAND driver will generate image checkpoint files if generate_checkpoint_images is enabled. At the start of the test run the driver will copy the current NAND image to a new file /tmp/synthnand.log.checkpoint, using the same base filename as for logfiles. If support for multiple logfiles is enabled then the current checkpoint file will be renamed in the same way and at the same time as the current logfile, and a new checkpoint file will be created using the current image data. Hence for any logfile it is possible to examine both the starting image and the final image (which may be the current one).

Bad Block Injection

There are two settings related to bad block injection: factory_bad and inject. The former is straightforward:

factory_bad 17 42 256 1019

This setting is used only when the NAND image file does not yet exist and the driver has to create a new one. The specified erase blocks are marked as factory-bad and hence unusable. The setting consists simply of one or more numbers, each in the range 0 to CYGNUM_NAND_SYNTH_BLOCK_COUNT-1. A maximum of 32 blocks can be marked factory bad.

Bad block injection is rather more complicated, in an attempt to make it sufficiently flexible for a variety of uses. The target definition file can contain one or more inject settings. There is a limit of eight erase definitions and eight write definitions, giving a maximum of sixteen bad block injection definitions. However some of these can use repeat, so the number of bad blocks injected during a test run is limited only by the size of the emulated device. Example settings are:

inject erase current after rand% 1024 erases
inject write current after rand% 100000 calls repeat
inject erase block 1 after 3 block_erases
inject write page 9860 after 1000 writes

The inject keyword should be followed by either erase or write. If erase then the bad block injection happens during a call to the driver's erase function, otherwise it happens during a call to write. This is followed by a block or page definition, the keyword after, an event counter, and a couple of optional flags.

The simplest block or page definition is the keyword current. This simply means that whichever block is being erased, or whichever block contains the page being written, will be marked bad. Typically this will be used for injecting random faults. If a given block is the subject of an above-average number of erase or write operations then it is more likely to be the current block in one of these definitions, so heavily-used blocks are more likely to fail.

The alternative to current is to list a specific block for an erase definition, or a specific page for a write definition. Blocks go from 0 to CYGNUM_NAND_SYNTH_BLOCK_COUNT - 1. Pages go from 0 to (CYGNUM_NAND_SYNTH_BLOCK_COUNT * CYGNUM_NAND_SYNTH_PAGES_PER_BLOCK) - 1. This can be particularly useful when testing higher-level code that uses certain blocks specially, for example for storing filesystem metadata. It can also be useful when attempting to repeat a test run using information from a logfile.

The fields immediately following the after keyword specify when the bad block injection should trigger. They consist of the optional keyword rand%, a count, and an event identifier. If rand% is not specified then exactly the specified number of events must occur before the bad block injection triggers. Otherwise some number between 0 and count-1 events must occur. The event identifier can be one of erases, writes, calls, block_erases, or page_writes. erases means the number of calls to the driver's block erase function. writes means the number of calls to the page write function. calls means the total number of read, write or erase calls. block_erases means erase calls for a specific block. Similarly page_writes means write calls for a specific page. block_erases and page_writes cannot be used together with current, only with a specific block or page.

So, consider the first example again:

inject erase current after rand% 1024 erases      

The driver will calculate a random number between 0 and 1023, say 427. Once there have been at least 427 erase calls the bad block injection will trigger. Since the definition is for erase current, the injection can happen immediately. Hence whichever block is specified during the 427th erase call will be marked bad and that erase call will fail with error code EIO.

Suppose that the definition used write current instead of erase current. The definition will still trigger during erase call 427, but it will not take effect immediately. Instead whichever page is the subject of the next write operation will be marked bad. Alternatively, suppose that the definition was for erase block 42 but call 427 was for block 512 instead. If some time after call 427 there was another erase call for block 42, that later erase call will fail and cause the block to be marked bad.

Now consider the next definition:

inject write current after rand% 100000 calls repeat

This event will trigger after between 0 and 99999 calls into the driver. These calls can be reads, writes, or erases. If the triggering call is a write then the block affected will be the one containing the page being written. Otherwise whichever page is the subject of the next write operation will be the one affected.

inject erase block 1 after 3 block_erases      

This definition will only ever affect block 1. The first two calls to erase block 1 will succeed. The third call will fail. Erase calls for any other block have no effect on this definition.

inject write page 9860 after 1000 writes

This definition will only ever affect page 9860. The definition will trigger after 1000 writes to any page. If the thousandth write happens to be for page 9860, it will fail. Otherwise the next write for page 9860 will fail, whenever that happens. If the definition specified event type page_writes instead of writes it would trigger only after 1000 writes to page 9860 instead of 1000 writes to any page.

Trigger definitions can be followed by two optional keywords. The first is repeat and indicates that the definition should trigger multiple times, not just once. repeat can only be used for block or page current, not for a specific block or page, since any given block can only fail once. The second keyword is disabled. This can be used to create a bad block injection definition which is not active by default but which can be enabled in the GUI interface.

All bad block injection definitions operate in parallel, not sequentially. It is possible for multiple definitions to trigger during a single call but a given block can only fail once.

Interactive Dialog

Although it is possible to change the logging and other settings between test runs by editing the target definition file, the package also provides a way of changing these during driver initialization time. If the command line option --nanddebug is used in addition to --io then the I/O auxiliary will pop up a dialog box allowing the various settings to be edited.

Figure 2. I/O auxiliary Dialog, Files

The dialog consists of a tabbed notebook with separate tabs for Files, Logging, and Errors. The default tab is Files. This shows the NAND device settings as configured via CDL options. If a current NAND image file exists then it can be deleted with a single click, allowing the test run to proceed with a new blank NAND device. The current image file can be saved away to an archive, either using a default name based on the current image name or by letting the user select a file. It is also possible to restore a previously-saved archive, again using the default name if that archive exists, or by selecting an alternative file. If any of the relevant files are changed outside this dialog then the refresh button forces the dialog to check the filesystem again. The various operations will be reported at the bottom of the dialog.

If the Logging tab is selected then the dialog changes to the following:

Figure 3. I/O auxiliary Dialog, Logging

There are separate checkbuttons for the six types of events that can be logged. It is also possible to change the current logfile and to edit the other settings related to logfiles.

The Errors tab is not yet implemented.

The dialog can be dismissed using the OK button or by hitting the ESC key. At that point the eCos application will resume running with the selected settings.