Testing Solana Programs: Essential Tools and Tips
A Comprehensive Guide to Unit and Integration Testing for Solana Development
GM! Welcome to the Torii Security blog, your go-to resource for all things related to Solana security and development. Here, we dive deep into Solana security research, provide meticulous code reviews (audits), and share our insights, research findings, and observations on Solana and the broader blockchain ecosystem.
Today, we're focusing on an essential aspect of Solana development: testing your on-chain programs. Testing is a critical part of the development process, ensuring that your programs run smoothly, securely, and efficiently in the fast-paced and high-performance environment that Solana offers. For this article, we'll focus on two essential types of tests: unit tests, integration tests, and E2E tests. We'll save the more security-oriented tests, such as fuzzing and invariants, for another post.
Unit tests
Unit testing in Solana is straightforward, primarily involving the execution of Rust unit tests. This method is perfect for validating individual components of your program, such as mathematical formulas. A key strategy in effective unit testing is designing or refactoring your program in a way that isolates these operations. By encapsulating math operations in separate functions, you can easily test them independently of your program's main logic, ensuring accuracy and simplifying debugging.
For example, consider the Orca Whirlpools repository, which demonstrates how to effectively unit test math operations. By separating the swap math logic into its own function, the tests focus solely on the correctness of these calculations without interference from other program elements. You can see how they achieve this in the swap_math.rs file.
Integration tests
Integration tests in Solana come in two main types:
Tests Using a Local Test Validator: These tests involve setting up a local test validator and executing tests against its RPC endpoint. This approach simulates a real Solana environment, making it ideal for end-to-end testing of your program's interactions with the blockchain. Tools like Anchor's TypeScript testing framework and Trident fall into this category. They enable you to verify that your program correctly handles various scenarios in a controlled, realistic setting.
Direct Interaction with Solana Bank / SVM: The second type of integration test allows you to interact directly with Solana's bank layer or SVM (Solana Virtual Machine), bypassing the need to run a local node or deal with non-relevant components like consensus. This method focuses purely on testing program logic and instruction execution, offering a more streamlined and efficient testing process. It is particularly useful for validating the core functionality of your program without the overhead of a full network simulation. Examples: official Solana Test suite, Mollusk, and LiteSVM.
Tests Using a Local Test Validator
When it comes to integration testing with a local test validator, two frameworks stand out: Anchor's TypeScript testing framework and Trident.
Anchor TypeScript Test Framework
The Anchor TypeScript test framework is the most popular and widely used tool for Solana integration tests. Its popularity is due to its extensive documentation and numerous examples, making it easy to get started and find support. Many Solana programs rely on it for their testing needs.
Pros:
Widely Used: With a large community and many examples available, it's easier to find help and resources.
TypeScript Familiarity: Using TypeScript can be a significant advantage, especially for full-stack engineers, as it helps them understand the protocol and how to interact with it.
Cons:
Potential Slowness: Tests can be slow to run, which might hinder rapid development cycles.
TypeScript Complexity: While TypeScript is powerful, defining types can sometimes be complex and non-intuitive, potentially leading to issues in test creation.
For more examples of testing using Anchor TypeScript tests, check out this great article by Helius.
Trident
Trident is a newer framework that allows you to write tests in Rust while it handles spinning up a local validator. This setup can be highly beneficial for those who prefer Rust and want a more seamless integration testing experience.
Pros:
Nice API: Trident offers a clean and intuitive API.
Built-in Fuzzing and Invariant Testing: These advanced features can help uncover edge cases and ensure your program behaves correctly under various conditions.
Automated Test Generation: Can generate personalized boilerplate code for both fuzzing and integration tests.
Cons:
New Framework: As a newer tool, Trident has fewer examples and less community support.
Limited Program Integration: Currently, you cannot add other programs to your test suite. For instance, if you're developing an NFT-related project and need to test interactions with Metaplex, Trident won't support this scenario yet.
Despite these limitations, Trident shows great potential and is worth keeping an eye on as it evolves. Examples can be found in the project GitHub repo.
Biggest Con of Using Local Test Validator
One of the major drawbacks of using a local test validator is the difficulty in mocking accounts and blockchain state. While it's possible to create a custom genesis with built-in accounts and programs for use in your test environment, this approach is limited compared to the methods presented later in this article. You can clone programs and accounts from the mainnet, but mocking specific accounts that belong to certain programs is more challenging. If your program integrates with other programs, it may be difficult to test that integration well.
Another significant issue is the inability to impact validator state easily. For example, modifying the block time to test behaviors related to elapsed time is not straightforward with this approach. This limitation makes it difficult to simulate certain conditions and thoroughly test all aspects of your program.
It's worth mentioning that both testing frameworks integrate seamlessly with the Anchor framework, but they encounter challenges when working with native Solana programs.
Direct Interaction with Solana Bank / SVM
Direct interaction with Solana's Bank or Solana Virtual Machine (SVM) is a faster alternative to using a local test validator. This approach abstracts away unnecessary layers, focusing solely on the execution of the bank or SVM without the need for cluster logic. As a result, tests run much faster and can be executed in parallel, enhancing efficiency and reducing development time.
Official Solana Test Suite
The official Solana Test Suite provides a robust framework for testing Solana programs with direct control over the bank, including block time manipulation and other parameters.
Pros:
Comprehensive Examples: Extensive examples and documentation, making it the standard way to test Solana programs.
Complete Control: Ability to manipulate the bank state, including block time, providing precise testing conditions.
Cons:
Boilerplate Code: Tests often require a significant amount of boilerplate code, making them cumbersome to write.
Dependency Management: Managing dependencies can be challenging, often leading to cargo dependency issues and difficulty finding compatible versions.
Great examples can be found in the Solana Program Library - for example, tests for the Token Program.
Litesvm
Litesvm is a fast and lightweight library designed for testing Solana programs. It creates an in-process Solana VM optimized for developers, significantly speeding up the test and compilation process compared to alternatives like solana-program-test and solana-test-validator. It also features an ergonomic API with sensible defaults and extensive configurability.
Pros:
Speed: Much faster than traditional methods due to its lightweight nature.
Ergonomic API: Provides a user-friendly API with configurable options.
Complete Control: Ability to manipulate the bank state, including block time, providing precise testing conditions. As an example check out this test - by using warp_to_slot, execution time (block time) can be modified.
Cons:
New Framework: As a newer tool, it has fewer examples and a smaller community.
Limited Resources: Not as many resources or community support compared to established frameworks.
Examples can be found in the official repo.
Mollusk
Mollusk is another tool that provides a test harness for Solana programs by directly invoking the executable program using the BPF Loader. This bypasses transaction sanitization and runtime checks, focusing purely on instruction processing.
Pros:
Direct Invocation / Speed: Bypasses unnecessary checks, providing a more direct and efficient testing process.
Bencher - Mollusk also offers a compute unit usage bencher for profiling a program's compute unit usage.
Complete Control: Ability to manipulate the bank state, including block time, providing precise testing conditions.
Cons:
New Framework: Similar to Litesvm, it is a newer tool with fewer examples and less community support. No examples were found except in the project repository.
Overall, direct interaction with Solana's Bank or SVM offers a more streamlined and faster testing process. While these methods have their own challenges, such as managing dependencies or dealing with fewer examples, they provide significant advantages in terms of speed, efficiency, and the ability to have more control over the testing environment like creating mock accounts or modifying test conditions like time.
Stay in Touch
Currently, we have a variety of tools available for testing Solana programs, but in practice, it remains challenging to achieve comprehensive and effective testing. For example, in the EVM world, Foundry is a well-documented and highly popular tool that simplifies the testing process significantly. We believe Solana deserves a similarly robust and user-friendly tool suite. While there are promising candidates emerging, we will have to see how these tools evolve to meet the growing needs of the Solana developer community.
What is your favorite way to test Solana programs? Reply in the comments.
Thank you for reading! To stay updated with our latest insights, research, and security tips, follow us on X and visit our website. If you need help or have any questions, feel free to reach out.
Stay safe, Mantlets. Cheers!
meltedblocks.