How To Read A Smart Contract Audit Report

Diligent smart contract audits are an essential part of crypto security. Getting the audit is only the first step, though. As a protocol owner, even if you receive an audit from one of the best smart contract auditors, you still have to understand their findings and implement any fixes. As an investor, how do you ensure the audit was well-conducted and thorough? Knowing how to read a smart contract audit report is key to staying safe.

How Does The Audit Process Work?

While the exact process will vary, audit firms will typically require some prerequisite material before beginning work. Typical items to ask for are:

  • A set scope of contracts to audit
  • A set codebase (code freeze)
  • High level protocol documentation
  • Medium-level functional specifications
  • Low-level code documentation and commenting

These items ensure that the auditors can work efficiently and spend the majority of their time finding bugs, not struggling to understand what the code is supposed to do.

During work, one or more auditors will review the codebase, compile their findings, and discuss internally with their team. Dialogue with the protocol team is common, especially to ensure complete understanding of what functions are supposed to do.

Depending on the audit firm and the scope of the audit, the auditors may split into two or three independent teams, which will each review the codebase and compile their own findings, then combine everything with their peers at the end. This redundancy helps ensure that there are no collective blind spots or false assumptions. All the findings will then be combined into the audit report.

When the audit is complete and the report is prepared, the auditors will discuss with the protocol team. Findings will be confirmed and acknowledged. Once a finding is confirmed, the protocol has to decide whether to fix the bug/mitigate the risk, or accept the risk without changes.

For example, an admin address with rights to withdraw funds from the contract does present a security risk, but that may be acceptable to the protocol team based on their business model.

At this point, depending on the individual contract with the audit firm, fixes for the audit findings may be re-audited and the fix incorporated into the report. When the final report is compiled, it may or may not be made public, depending on the protocol’s wishes.

What’s In An Audit Report?

Each audit report will vary, however most will have some variation on the following sections:

  • Executive summary
  • Scope
  • Manpower
  • Findings summary
  • Finding severity criteria
  • Detailed findings
  • Codebase maturity
  • Recommendations
  • Mitigations and fixes (not all reports)

Let’s dive in to each of these sections and see what the important takeaways are!

How To Read A Smart Contract Audit Report

Go through the report carefully, and know what’s important and what can be glossed over.

As I explain how to do that, we’ll reference the Trail of Bits audit report for Primitive as our example.

Executive Summary

As the name suggests, this is the overall summary of the report. The key factors you’ll find in a summary are:

  • What was audited?
  • Who audited it?
  • What were they looking for?
  • What did they find?

We’ll get into the details of each section and why they’re important in the detailed section.

Example executive summary from Trail of Bits

Scope and Resources

Smart contract audit firms do not have unlimited resources or unlimited time. Understanding how many resources were devoted to the effort helps benchmark the level of rigor possible. Audit reports will list the manpower used, their total time commitment, and any automated tools used to help speed the process. In the example report, we see this was conducted by two auditors over four weeks, and a full suite of automated tools were used.

Understanding the scope of the code being audited is also critical (See: Beanstalk DAO hack). Two auditors over two weeks looking over three contracts can get much more detailed than the same manpower spread over a 15-contract protocol. The report will also list the specific Github commit hash that was audited. This lets you see the exact code that was audited (if the code is public), and see what changes were made since then.

Repositories in scope, with commit hashes
Specific coverage notes

This section will also usually list the audit goals. Aside from the generalized mission of “find bugs”, protocols usually have something specific they’re worried about. One protocol may be particularly vulnerable to governance attacks, another may be sensitive to oracle manipulation or stale prices.

Specific goals of the Primitive audit

Codebase Maturity

The codebase maturity section is the most important section other than the actual findings.

Why? Codebase maturity and other judgements of development practice are wholly or partially subjective. Auditors have more freedom in this section to give their gut feeling or opinions. The judgement of an expert should not be taken lightly!

Actual audit findings are almost totally objective. Either there is an exploitable bug/extant security risk, or there is not. It may be however that no bugs are found during the review, but the code is sloppy, poorly commented, or otherwise not well taken care of. This signals that there is some underlying issue!

The code maturity section allows the auditors to raise these “soft” concerns as part of their review. This is not just nitpicking, as immature code repositories are often symptoms of a root cause. The development team may not be up to the task, or there may be high developer turnover, poor project management, or some other issue that bodes ill for the continued security of the protocol.

Example portion of a codebase maturity evaluation

Findings Summary

The findings section is the meat of the report. This is usually broken into two parts: a summary, and detailed technical description of the findings. Every reader, technical or nontechnical, should view the findings summary. This section is usually tabulated or written in plain English.

Portion of the Primitive findings summary

The summary table allows you to get a holistic sense of if there’s any overarching weaknesses in the protocol. In the Primitive report, most of the findings are in the categories of data validation and undefined behavior. This means that the developers have to put a concerted effort into validating all their inputs and outputs, and ensuring that the protocol arithmetic behaves in expected, well-defined ways.

Risk Scoring Criteria

The risk scoring criteria helps keep risk rankings consistent both within and across audits. This may be found in the report body, or as part of an appendix.

Trail of Bits standard Severity/Difficulty matrix

Detailed Findings

The detailed finding may not be of interest to nontechnical persons, such as users looking to invest in a protocol. Nonetheless, I’ll go over how to read it so you can be fully informed. The critical parts of a detailed finding are the severity, the difficulty, the proof of concept, and the mitigation.

The severity refers to the protocol’s impact if this finding were to be exploited. Audit firms have a matrix to determine this impact in a semi-objective manner. High severity usually means significant loss of funds, or cessation of protocol functionality, etc.

The difficulty refers to how difficult it is to exploit the vulnerability. A low difficulty would be a publicly-exposed mint or withdraw function, for instance. A high difficulty might require compromising a permissioned account, having very niche technical skills, or compromising an upstream protocol.

Finding TOB-PTV-1

The proof of concept or example exploit shows how it’s possible to exploit the vulnerability.

The mitigations or recommendations section of a finding will explain what can be done to fix the vulnerability.

Reading the mitigations section helps you understand whether this is an isolated issue, or something deeply entangled with the protocol’s core design philosophies.

Recommendations

In addition to the specific fixes recommended to patch vulnerabilities, the auditors may include a general recommendations section. This is an important one to pay attention to, since it can highlight deficiencies in the entire codebase or development process.

In the case of Primitive, the recommendations center around general good development practice, as well as a recommendation targeted at the protocol’s high amount of arithmetic findings.

Trail of Bits recommendations to Primitive

Issue Resolution

This may not be applicable to all audits. Sometimes the audit is one-time only, other times there will be a back-and-forth with the protocol team, and the fixes to acknowledged findings will be included in the audit report. The Trail of Bits audit for Primitive does not have any mitigations included, so let’s jump over to another report: Consensys Diligence for Fei Labs.

This report shows both the findings, and what was done to fix them.

Fei Labs Flywheel finding 1

When this information is present, it’s crucial to review it! You can verify that the fix was complete and thorough, not just a bandaid or token fix.

Wrapping Up

You don’t have to take a protocol at their word. Just because it’s been audited does not mean it’s secure. Not all auditors are created equal. Not all bugs may be discovered and raise findings. Not all findings are fixed – and not all fixes are adequate. Arm yourself with this knowledge, and do your due diligence. (Read: 5 Hazards To Watch For In DeFi)

Too many protocols use audits as marketing tools. Audits aren’t magic spells that cast out all bugs – they are an important piece of the security cycle. Read the reports, and see what the auditors actually found. The results may surprise you!

A final note: just because an audit found a few bugs does not mean a protocol is insecure. You hope the auditors find bugs, so you don’t find them when it’s deployed and handling funds! What you’re really interested in to determine security is the overall quantity and severity of the findings, the maturity of the codebase and development practices, and how the team handles fixes to the audit findings. Tons of high severity bugs, sloppy code practices, disputing legitimate findings, and bandaid fixes are all red flags.

Stay safe out there, anon.

Photo of author

Written By BowTiedPickle

Anonymous cartoon pickle inspired by BowTiedBull. Degen chemical engineer, moonlighting as a Solidity developer.

Disclosure

This article may contain links to third-party websites or other content for information purposes. BowTiedIsland may receive a commission at no cost to you if you purchase a product after clicking one of these links. The Third-Party Sites are not under the control of BowTiedIsland, and BowTiedIsland is not responsible for the content of any Third-Party Site. All information contained herein is the opinion of the writer and does not constitute financial advice. We aim to act as a neutral third party and aid in your research and analysis.


The Jungle


Crypto, Investing, and E-Commerce with BowTied Bull

The future is internet based, therefore we have a triangle based approach with crypto, e-commerce business making and Investing in traditional assets

The Culture War with BowTiedRanger

Whether you’re a political junkie or just interested in current events. 

You’ve come to the right place for analysis of the most relevant current events and political issues.

Fitness With BowTiedOx

BowTiedOx provides you a place to find all of his latest programs and guides.

Weekly newsletters that cover fitness, health, and mindset, all grounded in the fundamentals of physiology.

Media Production with BowTied Turkey and BowTied Tamarin

Video is no longer optional.

Don’t get left behind.

Your brand deserves professional videos to engage your audience.

Art & Graphic Design with BowTied Patriot

BowTied Patriot is a graphic artist who specializes in photography, mixed medium custom artwork, and NFT creation.

Join BowTiedPatriot as he dives into making Art in Web3.0 and The Metaverse.

Cooking with BowTiedOctopod

Learn secrets from a fine dining chef for maximum flavor and time-saving efficiency

Newsletters on Ingredients, Techniques and Flavor hacks that will have you eating better. We will never eat bugs!

Meme Warfare with DgenFren

Increase your online engagement, organically influence narratives, and build your online persona by using marketing that your target audience actually wants: memes.

Learn How to Sell with BowTiedSalesGuy

Sales is one of the most transferrable life skills, yet few know how to actually sell.

Traditional sales tactics don’t cut it in today’s hyper competitive world.

Learn the secrets from a Chad Salesman and change your Life forever.

Ecommerce with BowTiedOpossum

Learn the skills to start and build your first online business.

Want to build a business that travels with you?

Learn from an industry veteran that has worked on and with brands you already know.