Principles of Verifiable RTL Design

Lint Evaluation Script

The lint_eval.pl script at the following link runs small Verilog sample module files through lint tools.

It is an open-source Perl script that automates lint tool evaluations. The criteria in this script come from the [Bening and Foster 2001] book, Principles of Verifiable RTL Design 2nd Edition. The text of the book explains why these criteria are important, and the script supports automated test case no-quibble scores. It seems plausible that both the vendors and the lint users can do better with open-source test cases than with evaluation test cases under non-disclosure.

The lint evaluation perl script measures the quality of a linting tool by running small examples of Verilog modules through the tool. For most tests, if the tool can identify the error and take an "exit(1)," the tool passes the test for that example.

The Example and Section numbers identifying each test in the script refer to the Principles of Verifiable RTL Design book.

The lint_eval.pl scoring divides the tests into six categories:

  1. Case statements. eg. Fully-specified, explicit case and casex only, etc.
  2. No X state. eg. No X assignments, no X tests.
  3. Organization. eg. Storage elements only in libraries.
  4. Structure. eg. Latch. Feedback loop with no forgiveness pragma.
  5. Style. eg. Vendor-specific pragmas
  6. Typing. eg. Bit width, type must match between operands, operators.

From Olympic yacht racing, the lint_eval.pl scoring arithmetic copies a concept of assigning a premium to a top score. The theory is that if 1st place is worth far more than 2nd place, participants will put a lot more of effort to get a first place.

Within each of the six categories, the scoring starts at 100. A perfect score would be 600. For the first test within each category that the lint tool fails to diagnose a problem, the score loses 20 percent. Addition failures reduce the score by 1 percent.

This lint evaluation script is Open Software, and may be freely copied under the rules of the "zlib/libpng License" as described in http://www.opensource.org/licenses/index.html

    Copyright (c) 2003 Lionel Bening and Harry Foster

    This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software.

    Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions:

      1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.

      2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.

      3. This notice may not be removed or altered from any source distribution.