Diligent smart contract audits are an essential part of crypto security. Getting the audit is only the first step, though. As a protocol owner, even if you receive an audit from one of the best smart contract auditors, you still have to understand their findings and implement any fixes. As an investor, how do you ensure the audit was well-conducted and thorough? Knowing how to read a smart contract audit report is key to staying safe.
How Does The Audit Process Work?
While the exact process will vary, audit firms will typically require some prerequisite material before beginning work. Typical items to ask for are:
- A set scope of contracts to audit
- A set codebase (code freeze)
- High level protocol documentation
- Medium-level functional specifications
- Low-level code documentation and commenting
These items ensure that the auditors can work efficiently and spend the majority of their time finding bugs, not struggling to understand what the code is supposed to do.
During work, one or more auditors will review the codebase, compile their findings, and discuss internally with their team. Dialogue with the protocol team is common, especially to ensure complete understanding of what functions are supposed to do.
Depending on the audit firm and the scope of the audit, the auditors may split into two or three independent teams, which will each review the codebase and compile their own findings, then combine everything with their peers at the end. This redundancy helps ensure that there are no collective blind spots or false assumptions. All the findings will then be combined into the audit report.
When the audit is complete and the report is prepared, the auditors will discuss with the protocol team. Findings will be confirmed and acknowledged. Once a finding is confirmed, the protocol has to decide whether to fix the bug/mitigate the risk, or accept the risk without changes.
For example, an admin address with rights to withdraw funds from the contract does present a security risk, but that may be acceptable to the protocol team based on their business model.
At this point, depending on the individual contract with the audit firm, fixes for the audit findings may be re-audited and the fix incorporated into the report. When the final report is compiled, it may or may not be made public, depending on the protocol’s wishes.
What’s In An Audit Report?
Each audit report will vary, however most will have some variation on the following sections:
- Executive summary
- Scope
- Manpower
- Findings summary
- Finding severity criteria
- Detailed findings
- Codebase maturity
- Recommendations
- Mitigations and fixes (not all reports)
Let’s dive in to each of these sections and see what the important takeaways are!
How To Read A Smart Contract Audit Report
Go through the report carefully, and know what’s important and what can be glossed over.
As I explain how to do that, we’ll reference the Trail of Bits audit report for Primitive as our example.
Executive Summary
As the name suggests, this is the overall summary of the report. The key factors you’ll find in a summary are:
- What was audited?
- Who audited it?
- What were they looking for?
- What did they find?
We’ll get into the details of each section and why they’re important in the detailed section.
Scope and Resources
Smart contract audit firms do not have unlimited resources or unlimited time. Understanding how many resources were devoted to the effort helps benchmark the level of rigor possible. Audit reports will list the manpower used, their total time commitment, and any automated tools used to help speed the process. In the example report, we see this was conducted by two auditors over four weeks, and a full suite of automated tools were used.
Understanding the scope of the code being audited is also critical (See: Beanstalk DAO hack). Two auditors over two weeks looking over three contracts can get much more detailed than the same manpower spread over a 15-contract protocol. The report will also list the specific Github commit hash that was audited. This lets you see the exact code that was audited (if the code is public), and see what changes were made since then.
This section will also usually list the audit goals. Aside from the generalized mission of “find bugs”, protocols usually have something specific they’re worried about. One protocol may be particularly vulnerable to governance attacks, another may be sensitive to oracle manipulation or stale prices.
Codebase Maturity
The codebase maturity section is the most important section other than the actual findings.
Why? Codebase maturity and other judgements of development practice are wholly or partially subjective. Auditors have more freedom in this section to give their gut feeling or opinions. The judgement of an expert should not be taken lightly!
Actual audit findings are almost totally objective. Either there is an exploitable bug/extant security risk, or there is not. It may be however that no bugs are found during the review, but the code is sloppy, poorly commented, or otherwise not well taken care of. This signals that there is some underlying issue!
The code maturity section allows the auditors to raise these “soft” concerns as part of their review. This is not just nitpicking, as immature code repositories are often symptoms of a root cause. The development team may not be up to the task, or there may be high developer turnover, poor project management, or some other issue that bodes ill for the continued security of the protocol.
Findings Summary
The findings section is the meat of the report. This is usually broken into two parts: a summary, and detailed technical description of the findings. Every reader, technical or nontechnical, should view the findings summary. This section is usually tabulated or written in plain English.
The summary table allows you to get a holistic sense of if there’s any overarching weaknesses in the protocol. In the Primitive report, most of the findings are in the categories of data validation and undefined behavior. This means that the developers have to put a concerted effort into validating all their inputs and outputs, and ensuring that the protocol arithmetic behaves in expected, well-defined ways.
Risk Scoring Criteria
The risk scoring criteria helps keep risk rankings consistent both within and across audits. This may be found in the report body, or as part of an appendix.
Detailed Findings
The detailed finding may not be of interest to nontechnical persons, such as users looking to invest in a protocol. Nonetheless, I’ll go over how to read it so you can be fully informed. The critical parts of a detailed finding are the severity, the difficulty, the proof of concept, and the mitigation.
The severity refers to the protocol’s impact if this finding were to be exploited. Audit firms have a matrix to determine this impact in a semi-objective manner. High severity usually means significant loss of funds, or cessation of protocol functionality, etc.
The difficulty refers to how difficult it is to exploit the vulnerability. A low difficulty would be a publicly-exposed mint or withdraw function, for instance. A high difficulty might require compromising a permissioned account, having very niche technical skills, or compromising an upstream protocol.
The proof of concept or example exploit shows how it’s possible to exploit the vulnerability.
The mitigations or recommendations section of a finding will explain what can be done to fix the vulnerability.
Reading the mitigations section helps you understand whether this is an isolated issue, or something deeply entangled with the protocol’s core design philosophies.
Recommendations
In addition to the specific fixes recommended to patch vulnerabilities, the auditors may include a general recommendations section. This is an important one to pay attention to, since it can highlight deficiencies in the entire codebase or development process.
In the case of Primitive, the recommendations center around general good development practice, as well as a recommendation targeted at the protocol’s high amount of arithmetic findings.
Issue Resolution
This may not be applicable to all audits. Sometimes the audit is one-time only, other times there will be a back-and-forth with the protocol team, and the fixes to acknowledged findings will be included in the audit report. The Trail of Bits audit for Primitive does not have any mitigations included, so let’s jump over to another report: Consensys Diligence for Fei Labs.
This report shows both the findings, and what was done to fix them.
When this information is present, it’s crucial to review it! You can verify that the fix was complete and thorough, not just a bandaid or token fix.
Wrapping Up
You don’t have to take a protocol at their word. Just because it’s been audited does not mean it’s secure. Not all auditors are created equal. Not all bugs may be discovered and raise findings. Not all findings are fixed – and not all fixes are adequate. Arm yourself with this knowledge, and do your due diligence. (Read: 5 Hazards To Watch For In DeFi)
Too many protocols use audits as marketing tools. Audits aren’t magic spells that cast out all bugs – they are an important piece of the security cycle. Read the reports, and see what the auditors actually found. The results may surprise you!
A final note: just because an audit found a few bugs does not mean a protocol is insecure. You hope the auditors find bugs, so you don’t find them when it’s deployed and handling funds! What you’re really interested in to determine security is the overall quantity and severity of the findings, the maturity of the codebase and development practices, and how the team handles fixes to the audit findings. Tons of high severity bugs, sloppy code practices, disputing legitimate findings, and bandaid fixes are all red flags.
Stay safe out there, anon.
