The Challenge: Maintainable Upgradeability & Extensibility Within An Immutable Environment

Solidity development is one of those things that is simple to pickup, but will take years to master due to the paradigm shift and entirely new set of considerations you face as the developer or architect. The language and syntax itself is actually quite accessible, and most of the constraints like working within a 24kb bytecode footprint, or within the 1024 slot stack size are easy enough for devs entering the space to conform and adapt to.

But one sticky point of friction I'd run into whenever imagining anything more than a simple throwaway project was always that of upgradeability and extensibility within an immutable environment. If the project is a relatively simple ERC20/ERC721 project, or something that can be crammed into a single contract, then upgrades can likewise be a relatively straight forward problem to solve for by leveraging OpenZeppelin's Upgradeable contracts, or with existing EIPs like 1167, 1504 or 1822. But as the scope of a system expands, and its complexity increases, its increasingly less likely you'll be able to limit your service to a single contract, and for many good reasons also shouldn't want to even if you could. This then starts to require juggling a tangled web of contracts made upgradeable through one standard, and extensible at the discretion of the developer to enable extensibility. As you can imagine this can cause confusion and present hurdles for freshman blockchain devs, and hurdles beget bugs, which in this environment costs people money. Something never felt quite "right" about existing solutions... until I came across EIP2535 last year, and haven't been able to imagine complex projects except through its terms ever since.

Enter Diamonds: An Accessible Paradigm For Upgradeable/Extensible Smart Contract Development

I love trailofbits, but I do recall them taking issue with the new language of Diamonds, and its the one case I can think of where I have to state... I disagree. The new terms presented in ERC2535 are indeed new jargon a developer will now need to learn, yes. But previous solutions to the challenges solved by ERC2535 were no less jargon filled, and generally include a more technically oriented jargon that can be more off putting and often clumsy feeling. The terms "Diamond", "Facet", "Cut" and "Loupe", while bordering on skeuomorphic, do in fact help present the ideas and constructs of the model in a way that to me feels more accessible than past EIPs, and that accessibility is what I think gives this nascent standard its potency. Systems that once felt cumbersome to imagine the architecture for now reduce simply to the same modular oriented design developers of all background are used to, with simple add/upgrade/delete and reflection hooks available out of the box (or not, your choice, another beautiful design choice of this standard).

I got into programming professionally 15 years ago after a more senior programmer/nerd made me realize my cop out for not becoming one, that I'd always forget the syntax, was a silly objection when he shot back "so? you can Google the syntax, its the concepts that matter". I started applying to schools that night after the light bulb clicked and have never looked back. Diamonds now officially being a standard feels like a light bulb moment for the space by providing new, specifically non-technical terms for constructs that give newer blockchain devs that same reminder: these are the concepts that matter for upgradeable/extensible contracts in an immutable environment, with the underlying syntax being something you can always Google when you inevitably forget.

If you'd like to follow along with code as you read, a complete diamond contract can be found at mudgen's original implementation . This implementation is also leveraged within my Diamond Hardhat reference repo that extends it with Greeter/Farewell, Persistence1/Persistence2, and mock ERC20/721/1155 Facet examples, along with Hardhat tasks and tests for each.

So without further adieu, this is the new non-standard language devs diving into ERC2535 will need to internalize.

Diamond: The Shiny Entity People Are Attracted To

The diamond is both nothing, and everything, serving as the "thing" that people associate with your token/service, and might hold in their wallet. Its comprised solely of a constructor (which like other proxy solutions, is also the only constructor you can use, more on this below), and a fallback function that then delegates to whatever facet is registered for the requested function signature (if one is registered).

All diamonds must implement the IDiamond interface, which is comprised of an enum FacetCutAction exposing Add, Replace, Remove actions, a FacetCut struct (more on this after introducing Facets), and a DiamondCut event raised with each deployed iteration of the Diamond. Which if you read that right you are correct, means the Diamond contract itself actually has nothing to implement for this interface, with the real shiny magic taking place in the IDiamondCut interface (more on this below).

Facet: A Behaviour Defined Module

Facets are most simply thought of akin to packages in NPM (which is why I've started working on dfm/diamond.json attempting to make just that), allowing developers to approach their systems design modularly in a more intuitive way that feels not unlike dealing with node module development they may already be used to. Facets allow developers to separate their code into logical or behaviour defined partitions which has the dual benefit of making the code base easier to maintain, while also making the 24kb bytecode limit feel less like a limit, and more of a bumper helping you avoid a bad system design.

A facet itself can be nearly any smart contract you can imagine, with a few caveats worth keeping in mind as you begin. As alluded to above, similar to the limitation with other proxy based contracts, you won't be able to rely on constructors in your facets (there may be hacky ways around this, but as a rule just don't) and will need to leverage an initializer oriented approach, and one that also respects that each Facet clearly can't just call their initializer function initialize, so a naming convention should be adopted that keeps code understandable to avoid both collisions and developer confusion. As with any contract featuring an initializer, never leave your contract uninitialized given this is a well known, and entirely avoidable security risk. This same caveat lends itself to the subcaveat that all functions cut to the diamond will exist within a shared namespace, so likewise naming conventions should be adopted to avoid overly simple/generic function names that will be liable to collide with other Facets.

Another caveat to keep in mind as you develop your Facets is, like any proxy approach, your state will be separate from your logic, which can be a confusing concept at first but is made more simple to both conceptualize and manage with the Diamond standard's proposed AppStorage/DiamondStorage model (more on that below). This is one element that is most likely to require existing projects that might be interested in Diamonds to refactor their code, but the alterations are relatively straight forward (if you haven't already launched... nothing is straight forward with a production migration). This is also not something defined within the ERC2535 spec, so storage strategies will ultimately be up to the developer, and AppStorage/DiamondStorage isn't a requirement, but is a very good suggestion if you don't have an upgradeable storage strategy you prefer in mind IMO.

FacetCut & DiamondCut: The Struct & Event That Sculpts Your Diamond

Don't think of your diamond project as a simple, geometric gemstone as the name might have you. Instead, you're creating something more like a diamond sculpture, with an addition here, a deletion there, and upgrades all over the place as you shape your service to meet its objectives in an iterative way. The simple FacetCut struct as defined in the IDiamondCut interface underwrites each change to your Diamond, and reduces the complexities of managing contract versions to three simple fields: facetAddress, action, and functionSelectors[].

facetAddress, as you might have guessed, refers to the address of the facet you're acting on from your Diamond, which for Add and Replace actions is likely the Facet contract you've just deployed. action likewise aptly named refers to one of the FacetCutAction enum values: Add, Replace, Remove. As you might expect, once a function signature is added, a colliding one cannot be added, and requires you use Replace instead to update the referenced logic contract. You can alternatively Remove the registered function signatures, and then Add a Facet claiming them, but that's largely not going to be useful outside of maybe unit tests that sanity check everything works as expected (as is done in the diamond-hardhat repo tests). For Remove cuts, rather than referencing a facet address, simply pass in the 0 address to nullify those function signatures. functionSelectors is a bytes4[] value representing the encoded function selectors of the Facet, which you can get in ethers.js with facet.interface.encodeFunctionData('fooFunction').

The IDiamondCut interface defines a single function you're required to implement called diamondCut. This function takes in an array of FacetCut objects, an init address of the contract to execute the calldata on (if desired), and a calldata parameter that represents a function signature to call upon completion (useful for automatically initializing facets, though would require a separate cut per initializable Facet or the use of an initializer contract/function that then initializes each desired Facet). Being able to pass an array of FacetCuts allows you to save gas and Cut all Facet changes at once. If the action was an Add for example, and you included cuts for an ERC20 facet, a staking facet, and a burning facet, all functions of those facets will then be part of the Diamond with a single call to diamondCut. This is a wildly powerful pattern for change management, and one I'm genuinely excited to watch develop as it gets adopted and ideas shared around it (which I have 0 doubt it will be, regardless of how long I feel like an old man yelling at a cloud before it inevitably happens lol).

Full code of the IDiamondCut interface (also implementing IDiamond) plucked from mudgen's implementation:

/******************************************************************************\
* Author: Nick Mudge <nick@perfectabstractions.com> (https://twitter.com/mudgen)
* EIP-2535 Diamonds: https://eips.ethereum.org/EIPS/eip-2535
/******************************************************************************/


interface IDiamondCut {
    enum FacetCutAction {Add, Replace, Remove}
    // Add=0, Replace=1, Remove=2

    struct FacetCut {
        address facetAddress;
        FacetCutAction action;
        bytes4[] functionSelectors;
    }

    /// @notice Add/replace/remove any number of functions and optionally execute
    ///         a function with delegatecall
    /// @param _diamondCut Contains the facet addresses and function selectors
    /// @param _init The address of the contract or facet to execute _calldata
    /// @param _calldata A function call, including function selector and arguments
    ///                  _calldata is executed with delegatecall on _init
    function diamondCut(
        FacetCut[] calldata _diamondCut,
        address _init,
        bytes calldata _calldata
    ) external;

    event DiamondCut(FacetCut[] _diamondCut, address _init, bytes _calldata);
}

Loupe: Reflection As A Feature? Yes Please!

While the ERC2535 spec makes supporting the Loupe functions optional, with the limited effort/cost added I can't imagine why a developer wouldn't want to implement IDiamondLoupe.

The interface defines a Facet struct comprised of a facetAddress, and bytes4[] functionSelectors. Leveraging this data structure, it then exposes the following functions: facets(), facetFunctionSelectors(address), facetAddresses() and facetAddress(bytes4). These do what you might expect, but to be sure:

facets(): returns an array of Facet objects

facetAddresses(): similar to facets, except it returns address[] instead of Facet objects.

facetFunctionSelectors(address): returns a bytes4[] of function signatures associated with a given Facet address

facetAddress(bytes4): returns the Facet address for a given registered bytes4 function selector.

These 4 simple functions, that both can and should be cordoned into their own Loupe Facet making their overhead nearly non-existent, provide powerful options for the developer, not least of which is to ensure better test coverage through automated testing strategies leveraging the Loupe functions, or if naming conventions allow, to ensure initializable functions are immediately initialized by leveraging Loupe functions to check and run them. For the lightweight addition to your project, my suggestion would be to by default include them in any new Diamond project, and to keep them in mind as your project develops in order to leverage them where they can be helpful.

Storage: A Clean & Powerful Pattern Worth Consideration With/Without ERC2535

One of my favorite parts about the ERC2535 standard isn't even part of the standard, and that's the proposed patterns for handling Storage. Not being part of the standard is a good thing, because its a pattern almost worthy of its own standard, and it means developers can consider plugging it into their projects whether they're ERC2535 projects or not. In both the AppStorage and DiamondStorage patterns, the state itself lives within your Diamond (proxy) contract, which separates it from the logic contracts that will change over time, and both patterns leverage manipulation of memory slot positions to service global and contextual storage.

AppStorage: Diamond/App Scoped Shared Storage

The simplest type of storage to consider is the AppStorage pattern, which represents a shared state that will be accessible to all of your Facets through the clever use of memory slots, and the strategy of initializing your AppStorage struct at memory slot 0. This is then implemented simply by defining the app-level state you want in your AppStorage struct, accounting for all of the usual EVM state management caveats where order matters as you define/update it, and creating an AppStorageFacet that your Facet extends (remember order matters when it comes to Storage in Solidity, so this should be extended first to ensure the proper slot address is used) which exposes an appStorage() internal function your Facet can use to get the AppStorage struct.

An example of an AppStorage Contract might look like:

import "../structs/AppStorage.sol";

contract AppStorageFacet {
    AppStorage internal s;

    function appStorage() internal pure returns (AppStorage storage ds){
      assembly {
        ds.slot := 0
      }
    }
}

Which could then be used from within a Facet as simply as this:

import "../storage/facets/AppStorageFacet.sol";

contract PersistentFacet1 is AppStorageFacet {

    function setMessage(string memory msg_) external {
      AppStorage storage _s = appStorage();
      _s.message = msg_;
    }

    function getMessage() external returns (string memory){
      AppStorage storage _s = appStorage();
      return _s.message;
    }
}

DiamondStorage: Facet Or Featureset Scoped Storage

Similar to the AppStorage method above, DiamondStorage is likewise made up of both a Struct that defines the state you need to store, and a Facet that exposes a function to return the correct state from the right memory slot position. The difference between the two is that while AppStorage can reliably count on memory slot 0, DiamondStorage leverages keccak256 hashed keys that allow it to effectively namespace the state, making it accessible only to the contracts that extend the relevent DiamondStorage contract. This powerful pattern allows developers to either restrict data to within the context of a given facet, or share data between the relevant facets according to featureset needs by extending the storage contract from multiple facets. As with other elements of Diamond development, careful attention should be paid to naming conventions to avoid both collisions and confusion.

An example of a DiamondStorage Contract might look like:

import "../structs/ERC20FacetStorage.sol";

contract ERC20StorageFacet {

  function erc20Storage() internal pure returns (ERC20FacetStorage storage ds) {
      bytes32 position =  keccak256("diamond.erc20.diamond.storage");
      assembly {
          ds.slot := position
      }
  }
}

Which could then be used from within a Facet as simply as this:

import "../storage/facets/AppStorageFacet.sol";
import "../storage/facets/ERC20StorageFacet.sol";

contract ERC20Facet is AppStorageFacet, ERC20StorageFacet {

    function setSymbol(string memory symbol_) external {
      ERC20FacetStorage storage _s = erc20Storage();
      _s.symbol = symbol_;
    }

    function symbol() external returns (string memory){
      ERC20FacetStorage storage _s = erc20Storage();
      return _s.symbol;
    }
}

Note: while the example doesn't use it, the ERC20Facet above also imports/extends AppStorageFacet, and does so first. If your Facet needs to interact with AppStorage, remember order matters and to follow the same pattern.

The Shiny Paradigm Shift: Upgradeability & Extensibility By Default, or Diamond Driven Development

Raise your hand if your usual approach to a new project is to burn versions of a non-upgradeable contract against a local testnet node, and then at some future point when you're moving into or considering proper testnet releases, you implement upgradeability features then. This workflow can work great for all sorts of projects, and is certainly the norm at hackathons, but it feels wrong treating upgradeability and extensibility as separate afterthoughts instead of first-class mechanisms of your contract development workflow.

When working with Diamonds, you'll find you have no moment where your contract is not upgradeable/extensible, unless you specifically choose to by removing the DiamondCut facet once deployed. This is IMO a better workflow, and helps you arrive at a proper alpha/beta/production ready release by design, instead of only once returning to the production considerations later. With the right tooling to support development workflows, it feels perfectly natural to deploy your contracts, make your edits, and then cut your changes and continue without blowing away your test node or abandoning deployed contracts between changes like you might be used to during early development. I've begun to call this approach to the dApp SDLC "Diamond Driven Development", and now that I can't unsee the efficiency gains intend to explore it more, while building tooling to help support it (more to come on that in the dfm/diamond.json series).

The Missing Ingredient Of ERC2535: Community

If ERC2535/Diamond Driven Development is of interest to you, please consider signing up for this newsletter where I'll be aiming to collect thoughts/tutorials/projects related to Diamond Driven Development and Solidity development more broadly.

If extra excited by it, I'd love to chat with other Solidity devs toying with ERC2535 to learn more about what you're building with it. mudgen has been kind enough to make a Diamond Driven Development channel in the official ERC2535 Discord for just that. Feel free to drop in and share what you've been making, or ask whatever questions you want answered, or just to say hi :)