logo icon

DefenseCode ThunderScan® SAST Official OWASP Benchmark Results

by: DefenseCode |

DefenseCode is pleased to announce that the DefenseCode ThunderScan® SAST solution has been officially tested against the OWASP Benchmark project. OWASP Benchmark is a test suite designed to verify the speed and accuracy of software vulnerability detection tools. It is a fully runnable web app written in Java, supporting analysis by Static (SAST), Dynamic (DAST), and Runtime (IAST) tools that support Java. ThunderScan® scored the highest among a dozen of tested SAST tools with a final score of 46%. Considering the previous commercial average of 26% and the highest individual score of 32% (a balanced score calculated by subtracting False Positive Rate (FPR) from True Positive Rate (TPR)), ThunderScan® performance scores 15-25% better in detection than all other tested commercial SAST solutions, as presented in the graph below:

The benchmark results file produced by ThunderScan® can be downloaded here. ThunderScan® results parser is now a part of OWASP Benchmark test suite and can be seen here. The score can hence be easily verified without purchasing the solution, which is not the case for other tested SAST tools. To better understand the benchmark, consider the following. There are four possible test outcomes in the Benchmark:

  1. Tool correctly identifies a real vulnerability (True Positive – TP)
  2. Tool fails to identify a real vulnerability (False Negative – FN)
  3. Tool correctly ignores a false alarm (True Negative – TN)
  4. Tool fails to ignore a false alarm (False Positive – FP)

A lot can be learned about a tool from these four metrics. Consider a tool that simply flags every line of code as vulnerable. This tool will perfectly identify all vulnerabilities. But it will also have 100% false positives and thus adds no value. Similarly, consider a tool that reports absolutely nothing. This tool will have zero false positives, but will also identify zero real vulnerabilities and is also worthless. You can even imagine a tool that flips a coin to decide whether to report whether each test case contains a vulnerability. The result would be 50% true positives and 50% false positives. Benchmark provides a way to distinguish valuable security tools from these trivial ones.

ThunderScan® performance in other supported languages is equally impressive which is repeatedly proven by discovering critical vulnerabilities in popular open source applications as well through feedback given by our customers. DefenseCode will continue improving the accuracy and overall performance of our SAST solution.

 

Regards,

DefenseCode Team

Latest News

Close Bitnami banner
Bitnami