Harnessing Foundry Cheatcodes: JSON Report Generation with Solidity

July 16, 2023
Rss Fetcher

While navigating the complexities of smart contract development and testing, we developers are always on the lookout for innovative tools that can streamline our work. One such utility that has caught my attention is Foundry’s fuzz testing, or “fuzzing”. Although fuzzing is an excellent approach to catching unforeseen edge cases, it can often seem like a black box of mysterious magic from a high-level perspective.

Driven by my innate curiosity, I decided to delve deeper and explore how to capture select sets of temporarily stored variables within my tests as logged JSON reports. These reports, when effectively generated and utilized, can provide insightful information about our smart contracts’ performance and behavior. However, the process of generating these reports can be anything but straightforward.

In this article, we will explore the process of generating JSON reports using the Solidity development framework, Foundry. We will utilize Foundry cheat codes to serialize temporary values stored in memory during test runs and log them in local reports.

This article will be divided into three parts: test contract setup (Part 1), serialization setup (Part 2), and report recording (Part 3).

In the first part, we will set up our contract and the test contract’s deployment functions. The second part will involve Foundry’s vm.serailize() cheatcodes for creating a library for serializing several fixed-sized Solidity types. The third and final part will cover the report recording and verification using Foundry cheatcodes vm.readFile(), vm.parseJson(), and vm.writeJson(). Our test subject will be the SimpleERC20 contract.

Note: This article presumes that you have an intermediate understanding of the Solidity language, Foundry testing, and JSON serialization. If you need a refresher on these topics, we recommend going through the official Solidity documentation and the Foundry book’s Test section.

Without further ado, let’s dive in!

Project Config and Setup

Code: https://github.com/jasonjgarcia24/foundry-serialized-logs
Solidity version: 0.8.20
Forge version: forge 0.2.0 (58a2729 2023–05–16T00:03:39.980639280Z)
IDE: VSCode

To begin, we’ll create our Foundry environment by running the following commands.

$ forge init
$ forge install openzeppelin/openzeppelin-contracts

Your project’s structure and the Foundry configuration should now be similar to the below (btw, please feel free to remove that last line). The only notable difference you’ll likely see is the solc version within the foundry.toml configuration file. Please set your solc to match what I have below.

Now we’ll create our remappings. To do this, create a remappings.txt file within the base path of your project and ensure it matches the following:

forge-std/=lib/forge-std/src/

@openzeppelin/=lib/openzeppelin-contracts/

@base=src/

This will provide us with nice relative dependency remappings to help our imports be a bit more straightforward and obvious.

Next, remove all the Counter*.sol contracts. You can also completely remove the script directory if you like. We will not be using it here.

Now we can get started 😃.

Part 1 — Contract and Test Case Setup

Within our src folder, create SimpleERC20.sol.

// SPDX-License-Identifier: MIT
pragma solidity 0.8.20;

import {ERC20} from "@openzeppelin/contracts/token/ERC20/ERC20.sol";

uint256 constant _INITIAL_SUPPLY_ = 2008;

contract SimpleERC20 is ERC20 {
    constructor() ERC20("Simple ERC20", "SERC20") {
        _mint(msg.sender, _INITIAL_SUPPLY_);
    }
}

I won’t go too much into this contract as it’s a straightforward implementation of the Openzeppelin ERC20 contract.

To check that your project is setup correctly, you can run forge build to compile the SimpleERC20.sol file.

Next, within our test folder, create a SimpleERC20__test.sol file. Within this file, we’ll start off by creating the following:

SimpleERC20Test contract: our primary test contract that directs the necessary processes for SimpleERC20 deployment, testing, and report recording.
SimpleERC20ReportRecorder abstract contract: the contract that will contain the logic for report recording.
IDataStructs interface: the interface that establishes the data structs to record.

// SPDX-License-Identifier: MIT
pragma solidity 0.8.20;

interface IDataStructs {
    struct DealData {
        address receiver;
        uint8 giveAmount;
    }

    struct TokenData {
        address token;
        uint256 totalSupply;
        uint256 balanceOfCreator;
        uint256 balanceOfReceiver;
    }
}

abstract contract SimpleERC20ReportRecorder is IDataStructs {}

contract SimpleERC20Test is SimpleERC20ReportRecorder {
    function setUp() public {}

    function testSimpleERC20() public {}

    function _deploySimpleERC20() internal {}

    function _testDealSimpleToken() internal {}
}

Now we can get started with our testSimpleERC20() function.

function testSimpleERC20(address _token, DealData memory _dealData) public {
    // Setup.
    _deploySimpleToken(_token);

    // Test.
    TokenData memory _tokenData = _testDealSimpleToken(
        _token,
        _dealData.receiver,
        uint256(_dealData.giveAmount)
    );

    // Save report.
    // recordDebtData("simple-erc20", _dealData, _tokenData);
}

Alright, let’s break the updates down.

For this function, we’ll take in two fuzzed variables: the _token address and a DealData struct.

Setup 🏗: we set up our deployment of the SimpleERC20 contract within our test function _deploySimpleToken which takes in the address to use in creation of the SimpleERC20 contract.

Test 🧪: we establish our test for the SimpleERC20 contract with our _testDealSimpleToken() function. Given our primary purpose within this article is covering report generation, we’ll perform basic level tests. In short, we will be testing changes in token supplies to arbitrary accounts via the _testDealSimpleToken() function. For this we need the SimpleERC20 contract’s address to deploy, the receiver of an amount of tokens, and the amount of tokens to give to the receiver.

Save Report ✍🏽: now we call our recordDebtData() function to record the report data. Here we use the file name to save the report to and both the DealData and TokenData that will be recorded in our report. Note: this function will remain commented out until Part 3.

Now onto the deployment function.

address constant _CREATOR_ =
    address(uint160(uint256(keccak256("simple erc20 creator"))));

function _deploySimpleToken(address _token) internal {
    vm.startPrank(_CREATOR_);
    deployCodeTo("./out/SimpleERC20.sol/SimpleERC20.json", _token);
    vm.stopPrank();
}

We’ll start at the top again.

address constant _CREATOR_ =
    address(uint160(uint256(keccak256("simple erc20 creator"))));

Overall, there’s not much to take away from this line 🐒. It’s simply creating our _CREATOR_ address.

For this function, we introduce three new cheatcode functions: vm.startPrank(), deployCodeTo(), and vm.stopPrank().

vm.startPrank(_CREATOR_);
deployCodeTo("./out/SimpleERC20.sol/SimpleERC20.json", _token);
vm.stopPrank();

As noted in the official Foundry documentation, startPrank “Sets msg.sender for all subsequent calls until stopPrank is called.” The interface for both of these cheatcodes can be found within forge-std/Vm.sol.

The deployCodeTo() function is magic 🪄. This little helper will “pseudo-deploy a contract to an arbitrary address by fetching the contract bytecode from the artifacts directory.” In other words, this Foundry cheatcode allows us to simple provide the path to the artifacts file and the address to “deploy” the contract to. The deployCodeTo() cheatcode can be found in forge-std/StdCheats.sol.

You should now notice Undeclared identifier. errors when using the cheatcodes. To address this, we need to add to our imports and inheritance.

For cheatcodes vm.startPrank() and vm.stopPrank() we import the CommonBase contract. We then need to inherit this contract into SimpleERC20ReportRecorder. For the deployCodeTo() cheatcode, the StdCheats contract is imported and inherited by SimpleERC20Test. We inherit it here because we will only use its functions within the _deploySimpleToken() function within the SimpleERC20Test contract’s scope.

SimpleERC20__test.sol should now match the following:

// SPDX-License-Identifier: MIT
pragma solidity 0.8.20;

import {CommonBase} from "forge-std/Base.sol";
import {StdCheats} from "forge-std/StdCheats.sol";

interface IDataStructs {
    struct DealData {
        address receiver;
        uint8 giveAmount;
    }

    struct TokenData {
        address token;
        uint256 totalSupply;
        uint256 balanceOfCreator;
        uint256 balanceOfReceiver;
    }
}

abstract contract SimpleERC20ReportRecorder is
    CommonBase,
    IDataStructs
{}

contract SimpleERC20Test is
    StdCheats,
    SimpleERC20ReportRecorder
{
    address constant _CREATOR_ =
        address(uint160(uint256(keccak256("simple erc20 creator"))));

    function setUp() public {}

    function testSimpleERC20(address _token, DealData memory _dealData) public {
        // Setup.
        _deploySimpleToken(_token);

        // Test.
        TokenData memory _tokenData = _testDealSimpleToken(
            _token,
            _dealData.receiver,
            uint256(_dealData.giveAmount)
        );

        // Save report.
        recordDebtData("simple-erc20-deal", _dealData, _tokenData);
    }

    function _deploySimpleToken(address _token) internal {
        vm.startPrank(_CREATOR_);
        deployCodeTo("./out/SimpleERC20.sol/SimpleERC20.json", _token);
        vm.stopPrank();
    }

Continuing on, we establish the _tetsDealSimpleToken() function to execute our tests and provide us with our TokenData values.

import {CommonBase} from "forge-std/Base.sol";
import {StdCheats} from "forge-std/StdCheats.sol";
import {StdAssertions} from "forge-std/StdAssertions.sol";

import {_INITIAL_SUPPLY_} from "@base/SimpleERC20.sol";

import {IERC20} from "@openzeppelin/contracts/token/ERC20/IERC20.sol";

...

contract SimpleERC20Test is
    StdCheats,
    StdAssertions,
    SimpleERC20ReportRecorder
{
    ...

    function _testDealSimpleToken(
        address _token,
        address _receiver,
        uint256 _giveAmount
    ) internal returns (TokenData memory _tokenData) {
        // Instantiate contract.
        IERC20 _simpleToken = IERC20(_token);

        // Deal.
        deal(_token, _receiver, _giveAmount, true);

        // Get token data.
        _tokenData = TokenData({
            token: _token,
            totalSupply: _simpleToken.totalSupply(),
            balanceOfCreator: _simpleToken.balanceOf(_CREATOR_),
            balanceOfReceiver: _simpleToken.balanceOf(_receiver)
        });

        // Assert.
        assertEq(
            _tokenData.totalSupply,
            _INITIAL_SUPPLY_ + _giveAmount,
            "total supply should be increased"
        );

        assertEq(
            _tokenData.balanceOfCreator,
            _INITIAL_SUPPLY_,
            "balance of creator should be unchanged"
        );

        assertEq(
            _tokenData.balanceOfReceiver,
            _giveAmount,
            "balance of receiver should be equal to give amount"
        );
    }
}

From the top, we now see three new imports:

StdAssertions: the Foundry contract that defines our assertEq() functions.
_INITIAL_SUPPLY_: a familiar constant from within our SimpleERC20 file that defines our deployed contract’s initial token supply.
IERC20: the standard Openzeppelin interface that will allow us to interact with our SimpleERC20 contract’s functions when used with the _token address.

We can also see that our SimpleERC20Test contract now inherits the StdAssertions contract to give it access to it’s assertion functions.

Next, we notice our _testDealSimpletoken() function signature has changed to include the inputs and TokenData output.

function _testDealSimpleToken(
        address _token,
        address _receiver,
        uint256 _giveAmount
    ) internal returns (TokenData memory _tokenData) { ... }

Then we instantiate a temporary interface for our SimpleERC20 contract that was previously deployed to _token address.

// Instantiate contract.
IERC20 _simpleToken = IERC20(_token);

Now we use Foundry’s deal() function to deal _giveAmount of ERC20 tokens from the SimpleERC20 contract at _token address to the _receiver 💸. The final input parameter instructs the deal() function to also update the SimpleERC20 contract’s total token supply to reflect the dealing of tokens.

// Deal.
deal(_token, _receiver, _giveAmount, true);

After dealing tokens, we now populate the TokenData.

// Get token data.
_tokenData = TokenData({
    token: _token,
    totalSupply: _simpleToken.totalSupply(),
    balanceOfCreator: _simpleToken.balanceOf(_CREATOR_),
    balanceOfReceiver: _simpleToken.balanceOf(_receiver)
});

With a general understanding of the ERC20 standard, this initialization of the TokenData struct should be straightforward. If there are questions, please refer to the official ERC-20 token standard. This data will eventually be used in our report, so please make sure you understand it at a basic level.

To finish off this function, we build the actual test cases:

// Assert.
assertEq(
    _tokenData.totalSupply,
    _INITIAL_SUPPLY_ + _giveAmount,
    "total supply should be increased"
);

assertEq(
    _tokenData.balanceOfCreator,
    _INITIAL_SUPPLY_,
    "balance of creator should be unchanged"
);

assertEq(
    _tokenData.balanceOfReceiver,
    _giveAmount,
    "balance of receiver should be equal to give amount"
);

In summary, these minimal test cases will ensure that our dealt tokens are indeed distributed as expected.

Below is where we should be thus far:

// SPDX-License-Identifier: MIT
pragma solidity 0.8.20;

import {CommonBase} from "forge-std/Base.sol";
import {StdCheats} from "forge-std/StdCheats.sol";
import {StdAssertions} from "forge-std/StdAssertions.sol";

import {_INITIAL_SUPPLY_} from "@base/SimpleERC20.sol";

import {IERC20} from "@openzeppelin/contracts/token/ERC20/IERC20.sol";

interface IDataStructs {
    struct DealData {
        address receiver;
        uint8 giveAmount;
    }

    struct TokenData {
        address token;
        uint256 totalSupply;
        uint256 balanceOfCreator;
        uint256 balanceOfReceiver;
    }
}

abstract contract SimpleERC20ReportRecorder is CommonBase, IDataStructs {}

contract SimpleERC20Test is
    StdCheats,
    StdAssertions,
    SimpleERC20ReportRecorder
{
    address constant _CREATOR_ =
        address(uint160(uint256(keccak256("simple erc20 creator"))));

    function setUp() public {}

    function testSimpleERC20(address _token, DealData memory _dealData) public {
        // Setup.
        _deploySimpleToken(_token);

        // Test.
        TokenData memory _tokenData = _testDealSimpleToken(
            _token,
            _dealData.receiver,
            uint256(_dealData.giveAmount)
        );

        // // Save report.
        // recordDebtData("simple-erc20-deal", _dealData, _tokenData);
    }

    function _deploySimpleToken(address _token) internal {
        vm.startPrank(_CREATOR_);
        deployCodeTo("./out/SimpleERC20.sol/SimpleERC20.json", _token);
        vm.stopPrank();
    }

    function _testDealSimpleToken(
        address _token,
        address _receiver,
        uint256 _giveAmount
    ) internal returns (TokenData memory _tokenData) {
        IERC20 _simpleToken = IERC20(_token);

        // Deal.
        vm.label(_receiver, "SIMPLE_ERC20_RECEIVER");
        deal(_token, _receiver, _giveAmount, true);

        // Get token data.
        _tokenData = TokenData({
            token: _token,
            totalSupply: _simpleToken.totalSupply(),
            balanceOfCreator: _simpleToken.balanceOf(_CREATOR_),
            balanceOfReceiver: _simpleToken.balanceOf(_receiver)
        });

        // Assert.
        assertEq(
            _tokenData.totalSupply,
            _INITIAL_SUPPLY_ + _giveAmount,
            "total supply should be increased"
        );

        assertEq(
            _tokenData.balanceOfCreator,
            _INITIAL_SUPPLY_,
            "balance of creator should be unchanged"
        );

        assertEq(
            _tokenData.balanceOfReceiver,
            _giveAmount,
            "balance of receiver should be equal to give amount"
        );
    }
}

Alright, let’s give this a whirl 🌀!

$ forge test --match-test testSimpleERC20 -vv
[⠒] Compiling...
No files changed, compilation skipped

Running 1 test for test/SimpleERC20__test.sol:SimpleERC20Test
[PASS] testSimpleERC20(address,(address,uint8)) (runs: 256, μ: 342885, ~: 342870)
Test result: ok. 1 passed; 0 failed; finished in 311.38ms

Well, [PASS] equates to a job well done ✅ and the conclusion of Part 1! So far we’ve created our SimpleERC20 contract, the basic test cases, and we’ve set up the blueprints for Part 2 and 3.

Up next in Part 2, we’ll get familiar with a few of Foundry’s serialization functions and understand how to apply them for proper data formatting.

Part 2 — Serialization of Data

This part will be a nice reprieve from Part 1 and can be considered the calm before the storm that is Part 3 😶‍🌫️.

We’ll start by looking at serialization functions within the interface forge-std/Vm.sol:VmSafe.sol.

Notice the signatures are similar. According to Foundry’s documentation of the cheatcodes’ input arguments:

objectKey: A key for the object to which the value should be serialized to. This enables the user to serialize multiple objects in parallel
valueKey: A key for the value which will be its key in the JSON file
value: The value to be serialized

I think value is pretty straight forward, however I love visuals and I think these functions present the perfect opportunity for visuals:

                     // Level   | Object Key        | Value Key | Value     |
{                    //---------|-------------------|-----------|-----------|
  "level_1": {       // level 1 | base_obj          | level_1   | <level 2> |
    "greetings": {   // level 2 | child_obj_1       | greetings | <level 3> |
      "hi": "there", // level 3 | child_child_obj_1 | hi        | there     |
      "bye": "now"   // level 3 | child_child_obj_1 | bye       | now       |
    },               //         |                   |           |           |
    "nums": {        // level 2 | child_obj_1       | nums      | <level 3> |
      "num0": "0",   // level 3 | child_child_obj_2 | num0      | 0         |
      "num1": "1"    // level 3 | child_child_obj_2 | num1      | 1         |
    }
  }
}

The key, no pun intended, is to observer the matching object keys. Matching object keys will group multiple objects “in parallel” within their respective level. Value keys, represent the “key” in the key-value pair. So overall, a relatively nice and simple way to accommodate data serialization.

So now we’ll dive head first into our helper library.

import {Vm} from "forge-std/Vm.sol";

library SerializeHelper {
    // Cheat code address, 0x7109709ECfa91a80626fF3989D68f67F5b1DD12D.
    address constant VM_ADDRESS =
        address(uint160(uint256(keccak256("hevm cheat code"))));

    function serialize(
        uint256 _value,
        string calldata _objectKey,
        string calldata _valueKey
    ) public returns (string memory _serialized) {
        Vm vm = Vm(VM_ADDRESS);

        _serialized = vm.serializeUint(_objectKey, _valueKey, _value);
    }

    function serialize(
        address _value,
        string calldata _objectKey,
        string calldata _valueKey
    ) public returns (string memory _serialized) {
        Vm vm = Vm(VM_ADDRESS);

        _serialized = vm.serializeAddress(_objectKey, _valueKey, _value);
    }

    function serialize(
        string memory _value,
        string calldata _objectKey,
        string calldata _valueKey
    ) public returns (string memory _serialized) {
        Vm vm = Vm(VM_ADDRESS);

        _serialized = vm.serializeString(_objectKey, _valueKey, _value);
    }
}

There are a few things to take notice of here.

The first is the new import of the Vm interface from forge-std/Vm.sol.

The next is the VM_ADDRESS and its use for instantiating the Vm contract within our functions. This address is taken directly from the forge-std/Base.sol:CommonBase contract as shown below:

This allows us to have direct access to Foundry’s serialization functions and package them into helper functions for our specific use case.

With the prior explanations of the serialization cheatcodes’ input arguments and the Vm contract, we should have everything we need to understand what is going on within our SerializeHelper library. Therefore, given the object and value keys and the value of either type uint, address, or string, our library’s methods will leverage Foundry’s cheatcodes to consume the inputs and output the appropriate JSON serialized string for the information provided.

Okay! Two down, now the last to go.

Part 3 — Report Recording

Now we’ll head back into the SimpleERC20ReportRecorder contract. Here we’ll establish our logic for recording our JSON reports. Prior to heading into the code, we need to understand the basic process flow of fuzzed tests.

Given each test is uniquely run to completion, we can think of a single test within a fuzz sequence as an individual test that will run in its entirety, assuming no failures. Knowing this, will need to append to our historic JSON data set for each test report submitted. If this is not done correctly, we will end up with only the single, last data set. I hope this makes sense.

Continuing on with the good stuff, we’ll start with the addition of the recordDebtData() function.

abstract contract SimpleERC20ReportRecorder is CommonBase, IDataStructs {
    using SerializeHelper for *;

    function recordDebtData(
        string memory _outputFile,
        DealData memory _dealData,
        TokenData memory _tokenData
    ) public {
        _outputFile = string(
            abi.encodePacked("./report/", _outputFile, ".json")
        );

        // Read data.
        bytes memory _inputData = __readJson(_outputFile);

        // Parse data.
        string memory _outputData = __parseJson(
            _inputData,
            _dealData,
            _tokenData
        );

        // Write data.
        vm.writeJson(_outputData, _outputFile);
    }

    function __readJson(
        string memory _file
    ) private view returns (bytes memory _fileData) {}

    function __parseJson(
        bytes memory _prevData,
        DealData memory _dealData,
        TokenData memory _tokenData
    ) private returns (string memory _outputData) {}
}

This function will serve as the primary interface for triggering report recordings. The inputs, as previously noted in Part 1, are the _outputFile, _dealData, and the _tokenData.

The _outputFile specifies the name of our JSON data file. To create the relative path using the output file name, we use theabi.encodePacked() built-in function to convert the strings to a compressed byte array and then convert the bytes back to the resultant concatenated string. So if we reference the _outputFile used as the input in Part 1, our full file name would be ./report/simple-erc20.json.

_outputFile = string(
    abi.encodePacked("./report/", _outputFile, ".json")
);

Read data 👀: we use the __readJson() function to read previous fuzz tests’ JSON reports within the _outputFile. Our output here _inputData will need to be parsed out to individual discrete data types that align with the DealData and TokenData data structs.

// Read data.
bytes memory _inputData = __readJson(_outputFile);

Parse data 🧩: this section is our main bread and butter. This is where we parse the JSON data from the historic data as well as the current run’s DealData and TokenData data structs. In the __parseJson() function, we will bundle the data together into a serialized string that can be written directly to the output file.

// Parse data.
string memory _outputData = __parseJson(
    _inputData,
    _dealData,
    _tokenData
);

Write data 📓: writing a serialized string to a JSON file is very straightforward thanks to Foundry’s vm.writeJson() cheatcode. All we need to provide it with is the serialized string and output file name as shown below, which are _outputData and _outputFile respectively.

// Write data.
vm.writeJson(_outputData, _outputFile);

IMPORTANT: Before we continue, we need to make a configuration update. By default, your filesystem will likely block read-write access to Foundry’s cheatcodes. Therefore we need to add the fs_permission setting into our foundry.toml configuration. Going forward, your foundry.toml should contain the following settings:

[profile.default]
solc = "0.8.20"
src = "src"
out = "out"
libs = ["lib"]
fs_permissions = [{ access = "read-write", path = "./"}]

Alright, now back to our contract. Let’s dive a level deeper 🤿!

function __readJson(
    string memory _file
) public view returns (bytes memory _fileData) {
    // Read file. If file doesn't exist, return empty bytes.
    try vm.readFile(_file) returns (string memory _fileStr) {
        _fileData = bytes(_fileStr).length > 0
            ? vm.parseJson(_fileStr)
            : new bytes(0);
    } catch (bytes memory) {
        _fileData = new bytes(0);
    }
}

Here we have our __readJson() function. The goal of this function is to take in the _file path and read out the JSON parsed bytes array _fileData. To accomplish this we have two possible paths.

For the first attempt, we use Foundry’s vm.readFile() cheatcode. This cheatcode allows us to read the entire contents of our JSON data file as a string. If our file exists and this cheatcode succeeds, we will conditionally set the _fileData variable based off of the following two coditions:

_fileStr is not emtpy → assign _fileData to the JSON parsed bytes array of _fileStr
_fileStr is empty → assign _fileData to an empty bytes array

In the world of Solidity, to check a string’s length we must first convert the it to a bytes array and then use Solidity’s builtin length() function. So for this instance, if the return length is zero, the file is empty. If the length is not zero, we use Foundry’s vm.parseJson() cheatcode to return the entire file’s JSON data as a bytes array. To read more on how this cheatcode works, please check out the Foundry book.

_fileData = bytes(_fileStr).length > 0
    ? vm.parseJson(_fileStr)
    : new bytes(0);

Now for the second attempt. If the file does not exist, the vm.readFile() cheatcode will revert with a File doesn’t exist… failure. If this happens, we will directly set our _fileData to an empty bytes array.

} catch (bytes memory) {
    _fileData = new bytes(0);
}

And that’s it for reading the report’s historic data.

Now onto the fun! Let’s break down our new __parseJson() function. For this, we will do the following:

Update the __parseJson() function
Create a new interface for a ReportData data struct
Update the SimpleERC20ReportRecorder contract’s inheritance
Apply a using directive to attach our SerializeHelper library to all data types within the SimpleERC20ReportRecorder contract

So we definitely have some work to do! Below is the final form of the listed changes.

interface IReportDataStructs {
    struct ReportData {
        uint8 giveAmount;
        address receiver;
        uint256 balanceOfCreator;
        uint256 balanceOfReceiver;
        address token;
        uint256 totalSupply;
    }
}

abstract contract SimpleERC20ReportRecorder is
    CommonBase,
    IDataStructs,
    IReportDataStructs
{
    using SerializeHelper for *;

    ...

    function __parseJson(
        bytes memory _prevData,
        DealData memory _dealData,
        TokenData memory _tokenData
    ) private returns (string memory _outputData) {
        uint256 _dataLength = 32 * 6;
        uint256 _numElements = _prevData.length / _dataLength;
        bytes memory _chunk = new bytes(_dataLength);

        // Collect data.
        for (uint256 i; i <= _numElements; i++) {
            ReportData memory _reportData;

            // Get previous run's data.
            if (i < _numElements) {
                uint256 _offset = i * _dataLength;

                for (uint256 j; j < _dataLength; j++)
                    _chunk[j] = _prevData[_offset + j];

                _reportData = abi.decode(_chunk, (ReportData));
            }
            // Get new run's data.
            else {
                _reportData = ReportData({
                    receiver: _dealData.receiver,
                    giveAmount: _dealData.giveAmount,
                    token: _tokenData.token,
                    totalSupply: _tokenData.totalSupply,
                    balanceOfCreator: _tokenData.balanceOfCreator,
                    balanceOfReceiver: _tokenData.balanceOfReceiver
                });
            }

            // Package DealData object.
            _reportData.giveAmount.serialize("deal_data", "give_amount");
            string memory _dealDataObj = _reportData.receiver.serialize(
                "deal_data",
                "receiver"
            );

            _dealDataObj.serialize("level_2", "deal_data");

            // Package TokenData object.
            _reportData.balanceOfCreator.serialize(
                "token_data",
                "balance_of_creator"
            );
            _reportData.balanceOfReceiver.serialize(
                "token_data",
                "balance_of_receiver"
            );
            _reportData.token.serialize("token_data", "token");
            string memory _termsObj = _reportData.totalSupply.serialize(
                "token_data",
                "total_supply"
            );

            string memory _level2Obj = _termsObj.serialize(
                "level_2",
                "token_data"
            );

            // Package ReportData object.
            _outputData = _level2Obj.serialize(
                "level_1",
                string(abi.encodePacked("report_run_", vm.toString(i)))
            );
        }
    }
}

Okay… At first glance, there’s a lot going on here 😵‍💫!

Let’s break this down piece-by-piece again:

function __parseJson(
    bytes memory _prevData,
    DealData memory _dealData,
    TokenData memory _tokenData
) private returns (string memory _outputData) {

We begin with the SimpleERC20ReportRecorder contract, starting with the inputs. Here we remember that _prevData is the parsed JSON data bytes array gathered by our__readJson() function and _dealData and _tokenData are data structs containing this execution’s report data. The _outputData is the serialized string of the bundled report’s data for all tests.

uint256 _dataLength = 32 * 6; // 192
uint256 _numElements = _prevData.length / _dataLength;
bytes memory _chunk = new bytes(_dataLength);

The first three lines of this function initialize essential variables. _dataLength specifies the length of a single test’s saved data object. Below is an example of a single test run’s data:

192 bytes chunk:
0x0000000000000000000000000000000000000000000000000000000000000060000000000000000000000000ff26bee1dc7f7d9e25a90ddbd1afd7bd21fd61f500000000000000000000000000000000000000000000000000000000000007d8000000000000000000000000000000000000000000000000000000000000006000000000000000000000000015942335226c7b2d54a747cb5adc430dfc3629fc0000000000000000000000000000000000000000000000000000000000000838

Looking back at the DealData and TokenData data structs, we can see that we have 6 fields, and given Foundry’s vm.parseJson() cheatcode treats all uints as uint256, each one of our fields will take up 32 bytes.

struct DealData {
    address receiver;
    uint8 giveAmount;
}

struct TokenData {
    address token;
    uint256 totalSupply;
    uint256 balanceOfCreator;
    uint256 balanceOfReceiver;
}

We can then determine that the 192 byte chunk can be represented as 6 byte arrays with 32 total bytes:

6 - 32 byte groups
0x0000000000000000000000000000000000000000000000000000000000000060
0x000000000000000000000000ff26bee1dc7f7d9e25a90ddbd1afd7bd21fd61f5
0x00000000000000000000000000000000000000000000000000000000000007d8
0x0000000000000000000000000000000000000000000000000000000000000060
0x00000000000000000000000015942335226c7b2d54a747cb5adc430dfc3629fc
0x0000000000000000000000000000000000000000000000000000000000000838

We’ll hold onto this knowledge for when we conduct the actual parsing of these data sets.

Next, we can easily calculate the total number of test report objects within _prevData by dividing the length of _prevDataby _dataLength.

The third line initializes a bytes array _chunk with the length of our test’s data. This variable is going to help us break down our large data set into chunks of individual reports. This process will take some explaining. However before we continue forward within our contract, it’s time to explain that weird and redundant seeming ReportData struct.

interface IReportDataStructs {
    struct ReportData {
        uint8 giveAmount;
        address receiver;
        uint256 balanceOfCreator;
        uint256 balanceOfReceiver;
        address token;
        uint256 totalSupply;
    }
}

The first thing we probably notice is this struct has all fields combined from our DealData and TokenData structs. This is important because it’ll allow us to later decode our data using Solidity’s builtin abi.decode() function. The second thing that you may have not noticed is the order of the fields. You’ll notice that each field is in relative ascending order to the corresponding origination data struct’s fields. In other words, the fields of DealData are in ascending order to one another and the fields of TokenData are in ascending order to one another. In all honesty, I’m not completely sure why this sorting is needed. I assume the serialize() cheatcodes must perform some sort of sorting… 🤷.

Now onto less vague explanations! Here we’ll take a look at how this _chunk variable is used.

// Collect data.
for (uint256 i; i <= _numElements; i++) {
    ReportData memory _reportData;

    // Get previous run's data.
    if (i < _numElements) {
        uint256 _offset = i * _dataLength;

        for (uint256 j; j < _dataLength; j++)
            _chunk[j] = _prevData[_offset + j];

        _reportData = abi.decode(_chunk, (ReportData));
    }
    // Get new run's data.
    else {
        _reportData = ReportData({
            receiver: _dealData.receiver,
            giveAmount: _dealData.giveAmount,
            token: _tokenData.token,
            totalSupply: _tokenData.totalSupply,
            balanceOfCreator: _tokenData.balanceOfCreator,
            balanceOfReceiver: _tokenData.balanceOfReceiver
        });
    }
...
}

Soak that one up 🫠. Alright, we start by noticing the loop iterates inclusively over the _numElements. This loop allows sort through our report data and eventually serialize it in preperation for writing it to a file. By being inclusive to the _numElements, it allows us to include this current run’s data, as seen in the conditional’s else scenario.

Addressing the sorting of the _prevData bytes array, we must first notice the initiation of the ReportData struct _reportData. If we are in a condition where we are using previous report data (i.e. i < _numElements), we will need to establish our offset within the _prevData byte array. So for example:

i = 1; _offset = 1 * (32 * 6); // 192
i = 2; _offset = 2 * (32 * 6); // 384
i = 3; _offset = 3 * (32 * 6); // 576
...

Next, we temporarily copy each byte of the individual test report’s object into _chunk.

for (uint256 j; j < _dataLength; j++)
    _chunk[j] = _prevData[_offset + j];

This then allows us to use the ReportData struct to decode the _chunk to our _reportData struct using Solidity’s builtin abi.decode() function.

_reportData = abi.decode(_chunk, (ReportData));

To provide a visual of what is happening here, we’ll use Foundry’s amazing CLI tool chisel to break this down. We’ll use the 192 bytes array previously demoed.

First, we’ll individually convert the 32 byte sub-chunks:

$ chisel
➜
➜ uint256(0x0000000000000000000000000000000000000000000000000000000000000060)
Type: uint
├ Hex: 0x60
└ Decimal: 96
➜ address(0x000000000000000000000000ff26bee1dc7f7d9e25a90ddbd1afd7bd21fd61f5)
Type: address
└ Data: 0xff26bee1dc7f7d9e25a90ddbd1afd7bd21fd61f5
➜ uint256(0x00000000000000000000000000000000000000000000000000000000000007d8)
Type: uint
├ Hex: 0x7d8
└ Decimal: 2008
➜ uint256(0x0000000000000000000000000000000000000000000000000000000000000060)
Type: uint
├ Hex: 0x60
└ Decimal: 96
➜ address(0x00000000000000000000000015942335226c7b2d54a747cb5adc430dfc3629fc)
Type: address
└ Data: 0x15942335226c7b2d54a747cb5adc430dfc3629fc
➜ uint256(0x0000000000000000000000000000000000000000000000000000000000000838)
Type: uint
├ Hex: 0x838
└ Decimal: 2104

Then we’ll apply the abi.decode() function on the 192 byte array:

$ chisel
➜ 
➜ // Store 192 bytes array.
➜ bytes memory _chunk = hex"0000000000000000000000000000000000000000000000000000000000000060000000000000000000000000ff26bee1dc7f7d9e25a90ddbd1afd7bd21fd61f500000000000000000000000000000000000000000000000000000000000007d8000000000000000000000000000000000000000000000000000000000000006000000000000000000000000015942335226c7b2d54a747cb5adc430dfc3629fc0000000000000000000000000000000000000000000000000000000000000838"
➜ 
➜ // Set our ReportData struct.
➜ struct ReportData {
    uint8 giveAmount;
    address receiver;
    uint256 balanceOfCreator;
    uint256 balanceOfReceiver;
    address token;
    uint256 totalSupply;
}
➜ 
➜ // Parse our 192 bytes array using our ReportData struct.
➜ ReportData memory _reportData = abi.decode(_chunk, (ReportData))
➜ 
➜ // Display results.
➜ _reportData
tuple(60,ff26bee1dc7f7d9e25a90ddbd1afd7bd21fd61f5,7d8,60,15942335226c7b2d54a747cb5adc430dfc3629fc,838) = (
  ├ Type: uint
  ├ Hex: 0x60
  └ Decimal: 96

  ├ Type: address
  └ Data: 0xff26bee1dc7f7d9e25a90ddbd1afd7bd21fd61f5

  ├ Type: uint
  ├ Hex: 0x7d8
  └ Decimal: 2008

  ├ Type: uint
  ├ Hex: 0x60
  └ Decimal: 96

  ├ Type: address
  └ Data: 0x15942335226c7b2d54a747cb5adc430dfc3629fc

  ├ Type: uint
  ├ Hex: 0x838
  └ Decimal: 2104
)
➜

I hope that lines up for you and provides some clarity.

Alright then. Let’s take a moment to take in what we just did here! We just parsed individual data sets by making sense of a byte array data. We’re basically binary wizards 🧙!

Before continuing down our __parseJson() function, let’s flash back to our using directive set within our contract.

abstract contract SimpleERC20ReportRecorder is
    CommonBase,
    IDataStructs,
    IReportDataStructs
{
    using SerializeHelper for *;
    ...
}

The line using SerializeHelper for *; makes our library functions available to all types within our contract. Therefore, when calling the library functions, we can simply chain it to the end of a data type that matches the first input argument for that function.

Now back to our __parseJson() function.

Although this next section may look a bit intimidating, I promise is pretty much just copy-paste. Honestly, if you’re working with an AI buddy, it would probably just populate the rest for you.

// Package DealData object.
_reportData.giveAmount.serialize("deal_data", "give_amount");
string memory _dealDataObj = _reportData.receiver.serialize(
    "deal_data",
    "receiver"
);

_dealDataObj.serialize("level_2", "deal_data");

// Package TokenData object.
_reportData.balanceOfReceiver.serialize(
    "token_data",
    "balance_of_receiver"
);
_reportData.balanceOfCreator.serialize(
    "token_data",
    "balance_of_creator"
);
_reportData.token.serialize("token_data", "token");
string memory _termsObj = _reportData.totalSupply.serialize(
    "token_data",
    "total_supply"
);

string memory _level2Obj = _termsObj.serialize(
    "level_2",
    "token_data"
);

// Package ReportData object.
_outputData = _level2Obj.serialize(
    "level_1",
    string(abi.encodePacked("report_run_", vm.toString(i)))
);

Let’s start at the top again. So our first subsection in here is packaging up our DealData object. The fields for our DealData can be thought of as our third level in. Knowing this, we’ll use deal_data as our common objectKey for our DealData fields. The valueKey will be set specific to the field names, as can be noticed above. By using common objectKeys, we can be sure that the DealData fields will be bundled together within the same object. As previously mentioned, the values are chained with the library functions.

According to the Foundry book’s documentation, “the [vm.serialize() ] cheatcodes return the JSON object that is being serialized up to that point.” Therefore, we only need to assign the _dealDataObj to the second and final serialize() call for this object.

For the next line, we will establish our first serialization of the 2nd level. This level is where we will pair the two data structs together under the same run.

_dealDataObj.serialize("level_2", "deal_data");

For this our objectKey will be level_2 and our valueKey will be deal_data.

For the next subsection, we package up the TokenData object. I hope by now you can see this process is the same as the DealData. The changes are merely references to different object and value keys. However the process is the same for this object.

The next final step in this process is similar, however instead of bundling the individual data struct fields, it bundles the data objects into the base level (i.e. level_1). The other difference is the generation of a unique, sequential valueKey. By using Solidity’s type concatenation again, we can create our value keys for each test run’s data set.

string(abi.encodePacked("report_run_", vm.toString(i)))

Wow! I think we’re there! Now let’s set ourselves up to make sure this runs 🤞🤞🤞.

Run Everything

First off, we need to go back to our SimpleERC20Test contract and uncomment our call to recordData(). We’ll also add in bonus a convenience conditional statement to allow us to toggle recording on the fly through our CLI command. The vm.envOr() cheatcode allows us to use our the value of our RUN_REPORT environment variable to toggle report recording. If the environment variable is not set, it will default to no report.

// Save report.
if (vm.envOr("RUN_REPORT", false))
    recordDebtData("simple-erc20", _dealData, _tokenData);

Next, create a report folder in the project’s base path.

$ mkdir ./report

Now we’re good to go! We’ll start off by setting the RUN_REPORT environment variable to true and the number of fuzz runs to 3 with the FOUNDRY_FUZZ_RUNS environment variable.

Note: Foundry defaults this value to 256. I will admit that although more may be better in most scenarios, we have to also consider the time we have available to us to run these test scripts. This report generation process will increase the total execution time given we are juggling data and file system access throughout this report generation process. For our first PoC, it’s nice to conduct a test will a small amount of data so we can more efficiently verify the results.

Continuing on…

$ RUN_REPORT=true FOUNDRY_FUZZ_RUNS=3 forge test
[⠊] Compiling...
No files changed, compilation skipped

Running 1 test for test/SimpleERC20__test.sol:SimpleERC20Test
[PASS] testSimpleERC20(address,(address,uint8)) (runs: 3, μ: 828973, ~: 828143)
Test result: ok. 1 passed; 0 failed; finished in 11.21ms

And now open up our file:

$ code ./report/simple-erc20.json

{
  "report_run_0": {
    "deal_data": {
      "give_amount": 97,
      "receiver": "0x000000000000000000000000000000000000079b"
    },
    "token_data": {
      "balance_of_creator": 2008,
      "balance_of_receiver": 97,
      "token": "0x0000000000000000000000000000000000003304",
      "total_supply": 2105
    }
  },
  "report_run_1": {
    "deal_data": {
      "give_amount": 84,
      "receiver": "0x00000000000000000000000000000000000000b6"
    },
    "token_data": {
      "balance_of_creator": 2008,
      "balance_of_receiver": 84,
      "token": "0x00000000000000000000000000000000000004d2",
      "total_supply": 2092
    }
  },
  "report_run_2": {
    "deal_data": {
      "give_amount": 13,
      "receiver": "0x000000000000000000000000000000000000000C"
    },
    "token_data": {
      "balance_of_creator": 2008,
      "balance_of_receiver": 13,
      "token": "0x0000000000000000000000000000000000001A82",
      "total_supply": 2021
    }
  }
}

Finale!

Well, that was really something! I’m impressed if you stuck around. I hope you got something out of this and maybe it even answered some questions. If you have any suggestions, comments, or requests for clarification, please do reach out.

Thank you for reading and I hope it was fun.

If this made you want to throw money into a mostly empty pocket, please feel free to toss it here: 0x0b1928F5EbCFF7d9d2c8d72c608479d27117b14D.

If you’re a LinkedIn connection master, please reach out to me on LinkedIn with a note from this article if you’d like to connect.

Final Test Code

// SPDX-License-Identifier: MIT
pragma solidity 0.8.20;

import {Vm} from "forge-std/Vm.sol";
import {CommonBase} from "forge-std/Base.sol";
import {StdCheats} from "forge-std/StdCheats.sol";
import {StdAssertions} from "forge-std/StdAssertions.sol";

import {_INITIAL_SUPPLY_} from "@base/SimpleERC20.sol";

import {IERC20} from "@openzeppelin/contracts/token/ERC20/IERC20.sol";

interface IDataStructs {
    struct DealData {
        address receiver;
        uint8 giveAmount;
    }

    struct TokenData {
        address token;
        uint256 totalSupply;
        uint256 balanceOfCreator;
        uint256 balanceOfReceiver;
    }
}

interface IReportDataStructs {
    struct ReportData {
        uint8 giveAmount;
        address receiver;
        uint256 balanceOfCreator;
        uint256 balanceOfReceiver;
        address token;
        uint256 totalSupply;
    }
}

library SerializeHelper {
    // Cheat code address, 0x7109709ECfa91a80626fF3989D68f67F5b1DD12D.
    address constant VM_ADDRESS =
        address(uint160(uint256(keccak256("hevm cheat code"))));

    function serialize(
        uint256 _value,
        string calldata _objectKey,
        string calldata _valueKey
    ) public returns (string memory _serialized) {
        Vm vm = Vm(VM_ADDRESS);

        _serialized = vm.serializeUint(_objectKey, _valueKey, _value);
    }

    function serialize(
        address _value,
        string calldata _objectKey,
        string calldata _valueKey
    ) public returns (string memory _serialized) {
        Vm vm = Vm(VM_ADDRESS);

        _serialized = vm.serializeAddress(_objectKey, _valueKey, _value);
    }

    function serialize(
        string memory _value,
        string calldata _objectKey,
        string calldata _valueKey
    ) public returns (string memory _serialized) {
        Vm vm = Vm(VM_ADDRESS);

        _serialized = vm.serializeString(_objectKey, _valueKey, _value);
    }
}

abstract contract SimpleERC20ReportRecorder is
    CommonBase,
    IDataStructs,
    IReportDataStructs
{
    using SerializeHelper for *;

    function recordDebtData(
        string memory _outputFile,
        DealData memory _dealData,
        TokenData memory _tokenData
    ) public {
        _outputFile = string(
            abi.encodePacked("./report/", _outputFile, ".json")
        );

        // Read data.
        bytes memory _inputData = __readJson(_outputFile);

        // Parse data.
        string memory _outputData = __parseJson(
            _inputData,
            _dealData,
            _tokenData
        );

        // Write data.
        vm.writeJson(_outputData, _outputFile);
    }

    function __readJson(
        string memory _file
    ) private view returns (bytes memory _fileData) {
        // Read file. If file doesn't exist, return empty bytes.
        try vm.readFile(_file) returns (string memory _fileStr) {
            _fileData = bytes(_fileStr).length > 0
                ? vm.parseJson(_fileStr)
                : new bytes(0);
        } catch (bytes memory) {
            _fileData = new bytes(0);
        }
    }

    function __parseJson(
        bytes memory _prevData,
        DealData memory _dealData,
        TokenData memory _tokenData
    ) private returns (string memory _outputData) {
        uint256 _dataLength = 32 * 6;
        uint256 _numElements = _prevData.length / _dataLength;
        bytes memory _chunk = new bytes(_dataLength);

        // Collect data.
        for (uint256 i; i <= _numElements; i++) {
            ReportData memory _reportData;

            // Get previous run's data.
            if (i < _numElements) {
                uint256 _offset = i * _dataLength;

                for (uint256 j; j < _dataLength; j++)
                    _chunk[j] = _prevData[_offset + j];

                _reportData = abi.decode(_chunk, (ReportData));
            }
            // Get new run's data.
            else {
                _reportData = ReportData({
                    receiver: _dealData.receiver,
                    giveAmount: _dealData.giveAmount,
                    token: _tokenData.token,
                    totalSupply: _tokenData.totalSupply,
                    balanceOfCreator: _tokenData.balanceOfCreator,
                    balanceOfReceiver: _tokenData.balanceOfReceiver
                });
            }

            // Package DealData object.
            _reportData.giveAmount.serialize("deal_data", "give_amount");
            string memory _dealDataObj = _reportData.receiver.serialize(
                "deal_data",
                "receiver"
            );

            _dealDataObj.serialize("level_2", "deal_data");

            // Package TokenData object.
            _reportData.balanceOfReceiver.serialize(
                "token_data",
                "balance_of_receiver"
            );
            _reportData.balanceOfCreator.serialize(
                "token_data",
                "balance_of_creator"
            );
            _reportData.token.serialize("token_data", "token");
            string memory _termsObj = _reportData.totalSupply.serialize(
                "token_data",
                "total_supply"
            );

            string memory _level2Obj = _termsObj.serialize(
                "level_2",
                "token_data"
            );

            // Package ReportData object.
            _outputData = _level2Obj.serialize(
                "level_1",
                string(abi.encodePacked("report_run_", vm.toString(i)))
            );
        }
    }
}

contract SimpleERC20Test is
    StdCheats,
    StdAssertions,
    SimpleERC20ReportRecorder
{
    address constant _CREATOR_ =
        address(uint160(uint256(keccak256("simple erc20 creator"))));

    function setUp() public {}

    function testSimpleERC20(address _token, DealData memory _dealData) public {
        // Housekeeping.
        vm.assume(uint256(uint160(_token)) > 10);

        // Setup.
        _deploySimpleToken(_token);

        // Test.
        TokenData memory _tokenData = _testDealSimpleToken(
            _token,
            _dealData.receiver,
            uint256(_dealData.giveAmount)
        );

        // Save report.
        if (vm.envOr("RUN_REPORT", false))
            recordDebtData("simple-erc20", _dealData, _tokenData);
    }

    function _deploySimpleToken(address _token) internal {
        vm.label(_token, "FUZZED_SIMPLE_ERC20_TOKEN");
        vm.startPrank(_CREATOR_);
        deployCodeTo("./out/SimpleERC20.sol/SimpleERC20.json", _token);
        vm.stopPrank();
    }

    function _testDealSimpleToken(
        address _token,
        address _receiver,
        uint256 _giveAmount
    ) internal returns (TokenData memory _tokenData) {
        IERC20 _simpleToken = IERC20(_token);

        // Deal.
        vm.label(_receiver, "SIMPLE_ERC20_RECEIVER");
        deal(_token, _receiver, _giveAmount, true);

        // Get token data.
        _tokenData = TokenData({
            token: _token,
            totalSupply: _simpleToken.totalSupply(),
            balanceOfCreator: _simpleToken.balanceOf(_CREATOR_),
            balanceOfReceiver: _simpleToken.balanceOf(_receiver)
        });

        // Assert.
        assertEq(
            _tokenData.totalSupply,
            _INITIAL_SUPPLY_ + _giveAmount,
            "total supply should be increased"
        );

        assertEq(
            _tokenData.balanceOfCreator,
            _INITIAL_SUPPLY_,
            "balance of creator should be unchanged"
        );

        assertEq(
            _tokenData.balanceOfReceiver,
            _giveAmount,
            "balance of receiver should be equal to give amount"
        );
    }
}

Sources

Harnessing Foundry Cheatcodes: JSON Report Generation with Solidity was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.

Project Config and Setup

Part 1 — Contract and Test Case Setup

Part 2 — Serialization of Data

Part 3 — Report Recording

Run Everything

Finale!

Final Test Code

Sources

Previous Post

Next Post

Solutions

Regions Covered