moayuisuda

Recently, I created a small dapp project to reinforce my learning of smart contracts and also tried to use AI to enhance the customization and fun of NFTs. This article mainly records some key steps in the process.

Brain Storm#

I usually start with brainstorming, imagining the final effect, and then working backward to implement it.

1. Use AI to generate avatars. Simply generating avatars is a bit boring; the generated avatars can be related to the state of the real world. I want a block to anchor the AI's black box output and the feeling of the real-world state.
2. Add a card-drawing element, where for AI painting, this "card" is defined as "painting keywords."
3. The generated avatars can be minted as NFTs. Simply minting is also a bit boring; it shouldn't just be an image on the chain; it can have more of a "genuine" feel. I am willing to spend money on genuine games and artworks, partly to support the authors and largely because of the subsequent supporting services. In addition to the inherent trading characteristics of NFTs, users after minting should have more additional extended functions.

Let's briefly determine the user flow for the first version:

1. After entering, click the generate button (with optional configurations) to generate the avatar image.
2. After generation, you can choose one to mint as an NFT.
3. If you want to mint, click to connect your wallet and then click mint.
4. After minting is complete, minting information will be recorded as a basis for providing subsequent services (e.g., generating based on the seed of the minted image). At the same time, a certain number of points will be rewarded, and a painting keyword will be drawn.
   

Effect Preview#

GPU servers are very expensive. To prevent malicious DDoS attacks, the online trial is temporarily open only to a few friends. Here are some screenshots to see the process:

Initial state, the user has not linked a wallet, and only the "flower" keyword is available:

Click "make a wish" to generate an avatar.
Each avatar's result is somewhat related to the time and place of generation. For example, if you generate it during the day in Japan, the character in the avatar will be dressed in a kimono during the day:

After obtaining the image, if you want to mint, you need to connect your wallet:

After successfully connecting the wallet and minting, a drawn painting keyword will be gifted:

Returning to the homepage, you will find that you can generate avatar images based on the minted images, and additional usable keywords have been added:

Let's try generating based on the previous avatar using the new keyword:

The new keyword "Mononoke Hime" comes from "Princess Mononoke," and the generated avatar will also have the style of the princess:

The preview is complete. Now let's talk about how each part of the application is implemented.

AI Avatar Generation Service#

The currently most popular open-source AI painting generation project is stable-diffusion-webui (hereinafter referred to as sd-webui). After deployment, it can directly start a local web UI service or run in API mode, exposing interfaces for other services to call. You just need to add the --api parameter when starting.
Clone the repository and install it. Run the command under the sd-webui repository:
./webui.sh --api
or
./webui.sh --nowebui
The difference is that --api provides API interface services while still starting the original local web UI service, whereas --nowebui only provides interface services.

After successful startup, open this webpage:
http://127.0.0.1:7860/docs, and you should see the corresponding API documentation:

Then you can call the sd-webui interface like calling a normal API. For example, the most commonly used text2img function:

The image resource returned by the interface is a base64 string, which is convenient for front-end processing. Additionally, you can use the randomness of the black box noise of the sd model to hash the base64 string or directly hash the image seed as a pseudo-random seed for other functions like lottery.
If cross-origin requests are needed after deployment, add the --cors-allow-origins parameter when running "webui.sh".

※ If there are too many issues during the installation process, you can try using this docker image directly: https://hub.docker.com/r/kestr3l/stable-diffusion-webui

Contract Writing#

The functions needed for the contract:

1. Basic NFT functions (minting, transferring, associating tokenURI).
2. Record all NFT tokenIds owned by a user.
3. Points system (to be used in subsequent services).

This time, I chose to write the smart contract based on the ERC721Enumerable base contract. The contract code is as follows:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.7;

import "@openzeppelin/contracts/token/ERC721/extensions/ERC721Enumerable.sol";
import "@openzeppelin/contracts/access/Ownable.sol";
import "hardhat/console.sol";

error Insufficient_Mint_Value();

contract HimeNft is ERC721Enumerable, Ownable {
    uint256 private constant MINT_PRICE = 0.1 * 1e18;
    uint256 private _tokenId = 0;
    mapping(uint256 => string) private _tokenURIs;
    mapping(address => uint256) private _points;

    constructor() ERC721("HimeNft", "HN") {}

    function mintNft(
        string calldata tokenURI
    ) public payable returns (uint256) {
        if (msg.value < MINT_PRICE) revert Insufficient_Mint_Value();

        _safeMint(msg.sender, _tokenId);
        _setTokenURI(_tokenId, tokenURI);
        _tokenId += 1;
        _points[msg.sender] += 10;
        return _tokenId;
    }

    function withDraw() public onlyOwner {
        (bool callSuccess, ) = payable(msg.sender).call{
            value: address(this).balance
        }("");
        require(callSuccess, "call failed");
    }

    function _setTokenURI(
        uint256 tokenId,
        string memory _tokenURI
    ) internal virtual {
        require(
            _exists(tokenId),
            "ERC721URIStorage: URI set of nonexistent token"
        );
        _tokenURIs[tokenId] = _tokenURI;
    }

    function tokenURI(
        uint256 tokenId
    ) public view virtual override returns (string memory) {
        _requireMinted(tokenId);

        return _tokenURIs[tokenId];
    }

    function getTokenCounter() public view returns (uint256) {
        return _tokenId;
    }

    function getPoints(address owner) public view returns (uint256) {
        return _points[owner];
    }

    function getOwnedTokens(
        address owner
    ) public view returns (uint256[] memory) {
        uint256 tokenCount = balanceOf(owner);
        uint256[] memory result = new uint256[](tokenCount);
        for (uint256 i = 0; i < tokenCount; i++) {
            result[i] = tokenOfOwnerByIndex(owner, i);
        }
        return result;
    }
}

※ This is just a simple implementation. For future functional upgrades, it is better to use a proxy contract to mediate interactions between the application and the contract.

Here is some test code for local testing:

const { assert, expect } = require("chai");
const { deployments, ethers, getNamedAccounts } = require("hardhat");

describe("HimeNft", async () => {
  const MINT_VALUE = ethers.utils.parseEther("0.1");
  const INSUFFICIENT_MINT_VALUE = MINT_VALUE.sub(1);
  const TEST_TOKENURI = "TEST_TOKENUR";
  let deployer, nft;

  beforeEach(async () => {
    await deployments.fixture();
    deployer = (await getNamedAccounts()).deployer;
    nft = await ethers.getContract("HimeNft", deployer);
  });

  it("initial token count is 0", async () => {
    const counter = await nft.getTokenCounter();

    assert.equal(counter, 0);
  });

  it("revert on mint value is insufficient", async () => {
    expect(
      nft.mintNft(TEST_TOKENURI, {
        value: INSUFFICIENT_MINT_VALUE,
      })
    ).to.be.revertedWith("Insufficient_Mint_Value");
  });

  it("add correct points after mint", async () => {
    await nft.mintNft(TEST_TOKENURI, {
      value: MINT_VALUE,
    });
    const points = (await nft.getPoints(deployer)).toString();

    assert.equal(points, "10");
  });

  it("get correct tokenURI and token list", async () => {
    const TEST_TOKENURI0 = "TEST_TOKENURI0";
    const TEST_TOKENURI1 = "TEST_TOKENURI1";

    await Promise.all([
      nft.mintNft(TEST_TOKENURI0, {
        value: MINT_VALUE,
      }),
      nft.mintNft(TEST_TOKENURI1, {
        value: MINT_VALUE,
      }),
    ]);

    const list = (await nft.getOwnedTokens(deployer)).toString();
    const tokenURI0 = await nft.tokenURI(0);

    assert.equal(tokenURI0, TEST_TOKENURI0);
    assert.equal(list, "0,1");
  });
});

But you may notice that there seems to be no logic for "drawing a painting keyword."
Remember the earlier mention of "directly hashing the image seed as a pseudo-random seed for other functions like lottery"? To save storage space, I store the image seed in the NFT's metadata and map the seed to a painting keyword. This way, the minting process can also complete the "drawing of a painting keyword," while saving gas.
For example, the simplest case, I have a map like this:

{
  "royal": 0,
  "cute": 1,
}

I process the seed using modulo. For example, if the seed is 334451, and we set the divisor to 10, taking the modulo mod(334451,10) gives a result of 1, which corresponds to the drawn result being the "cute" keyword. Of course, this mapping algorithm can be more complex, even configuring various probabilities.
For sufficient transparency, this map method should be uploaded to the blockchain and verified.

The contract also defines a "_points" map to store points, which will be used as tokens for other services later (for simplicity, I did not use an additional ERC20 contract to implement tokens; if you want it to be a more "standard" token, you can implement it separately with ERC20, or change this contract to ERC1155 for multi-token management).

Building the Frontend Interface#

This time, I used next.js + tailwind + react-moralis + ethers.js to build and deploy the frontend service.
I won't elaborate here; the code is available at https://github.com/moayuisuda/HimeAvatar.

It is worth noting that the useWeb3Contract hook from react-moralis supports batch calls:
You can directly call contract methods:

const { runContractFunction } = useWeb3Contract({
  abi,
  contractAddress: address[chainId as keyof typeof address],
  functionName: "mintNft",
  params: { tokenURI: "TOKENURL" },
});

runContractFunction()

You can also define the contract's ABI, address, and other basic information first, then call specific methods:

const { runContractFunction } = useWeb3Contract({
  abi,
  contractAddress: address[chainId as keyof typeof address],
});

runContractFunction({
  params: {
    functionName: "mintNft",
    params: { tokenURI: "TOKENURL" },
  },
})

With that, the construction of each part is complete. You can package these services into a docker for easy one-click deployment on the server.

Todo#

• Use OpenAI's SDK to build a coherent worldview and dialogue (I personally recommend using the Python version of the SDK; I find the Node.js version a bit awkward), turning the simple avatar creation into character creation.
• Support creators to upload seed images and generate Lora models, producing images and sharing profits with creators.
• Build additional service stores for points tokens.

Gift#

The models used in the application are personally refined by me. The base model of the large model may have copyright issues, so generating avatar images is completely free.
Here are some nice avatars generated during previous refining sessions; feel free to use them if you like: