Engineering

How RugCheck's RPC Woes Defined FluxRPC

That little raccoon is hungry. Running RugCheck at scale taught us exactly what we wanted from a Solana RPC, and what we never wanted to pay for again.

· 6 min read

RugCheck has extremely heavy RPC usage. That little raccoon might be cute, but it's hungry. It consumes data like you wouldn't believe.

We knew we wanted to build an RPC. However, it was the frustrations RugCheck encountered with Solana RPCs that shaped how FluxRPC would be better. We're sure there are other developers out there, that have the same frustrations. So we thought we'd take a moment to share how we built fluxRPC differently, so that you don't have to go through the same woes.

At Any Cost, No Vague Billing Practices

The biggest problem we had, was that it was hard to keep RPC costs reasonable as the number of users scaled. As RugCheck started to become popular, our RPC bills increased, but it was hard to work out what exactly to optimize to keep them down. The main culprit was unclear "credit" or "compute unit" (CU) based billing.

If you look at the Solana RPC methods documentation, there are a lot of obvious optimizations available to save bandwidth by limiting the size of the RPC response. These are filters like 'memcmp' or 'DataSize', as well as parameters like DataSlice, that can vastly reduce the response size.

While the amount of data RugCheck consumes is a concrete and quantifiable optimization target, it didn't map linearly to decreased costs. The RPC call usually cost the same number of credits, so why bother with these optimizations? So our first goal for FluxRPC became bandwidth-based billing, so that RPC call optimizations would directly translate into lower costs for the people that build things efficiently.

Furlock Sholmes squinting through a magnifying glass at an invoice that unrolls across the floor, every line item a question mark

Engineering is much easier when you have one clear thing to optimize for! To help new Solana developers though, we cover some of these optimizations in the FluxRPC Developer Guides.

Yellowstone gRPC Data Streaming has to be Great

We had a similar problem with Yellowstone gRPC streaming. Streaming is the only practical way for RugCheck to get recently minted tokens, changes to liquidity, and pool creation the moment they happen. It's simply not practical to respond to every request for a RugCheck token report, by fetching all the required data on-chain via RPC calls. In practice, we need to already have most of the data required by a token report locally, in order to respond within a reasonable amount of time.

Here again, we encountered complex and vague billing practices. At different RPC providers, Yellowstone was either:

  1. Gatekept behind an expensive plan that contains tons of features or capacity we wouldn't need.
  2. A costly add-on with separate limitations, on top of the monthly cost.
  3. A costly dedicated server that was legitimately good, but oversized for our needs.
  4. Had unclear pricing for RPC calls
ProviderIssueDetails
Helius1,4LaserStream gRPC starts at USD 499
ChainStack2,4USD 449 as an add-on
Alchemy4Yellowstone is USD 0.073 per GB on top of separate RPC pricing
QuickNode1,2,4Scale Plan at USD 424 per month
Triton-requires USD 125 deposit, USD 0.08 per GB
FluxRPC-streaming starts at USD 15, USD 0.06 per GB
Entry price for Yellowstone gRPC streaming, where the cheapest plan elsewhere starts in the hundreds

Yellowstone gRPC has very powerful subscription management features for its data stream, but again, actually applying those optimizations had little effect on our costs. Either we had to pay for a premium plan anyway, that had features or capacity we would never use, or Yellowstone was an add-on with a (high) flat fee.

This is why we include Yellowstone gRPC in all paid FluxRPC plans, and the bandwidth for Yellowstone costs the same as for anything else (USD 0.06 per GB), to make optimization at scale easy. It's even 15$ for the first month, so you can make sure it works for you, which is a legitimately good deal. We really want to reduce the barrier to using Yellowstone gRPC.

Customer Support has to be Developer-Oriented and Fast

We had some bad experiences getting support for RPC issues when they came up. Even getting a next-day confirmation that the problem wasn't our-side would have saved us a lot of effort. Actually reaching an engineer from customer support is such a common hurdle, we write comics dreaming about it.

So we built our customer support differently: our customer support requests don't just reach an engineer. They reach ALL our engineers, the moment you submit it. Then one of us picks it up ASAP and contacts you.

Five copies of Furlock Sholmes racing in from every side toward a single support ticket

Caching

Many Solana applications follow the same pattern: Yellowstone data streams populate a local cache or DB, and where possible, you use your local data instead of making RPC calls. This avoids the cost and round-trip time to the RPC, and scales better as your user base grows. However, it's a bit of a pain to build that patten over and over in every application. Our experience doing this for RugCheck is also what led us to develop Lantern!

Furlock Sholmes walking a data centre aisle, holding up a lantern lit with the FluxRPC mark

RugCheck taught us that while Yellowstone had powerful subscription features, it was sending through much more data than it needed to, to keep our local cache up-to-date. So we wanted Lantern (our standalone Solana caching solution) to be able to:

  1. Minimize data usage to a small fraction of Yellowstone, 10x or better!
  2. Answer all RPC requests, handling cache hits and misses transparently.
  3. Store only the Solana accounts you need, with flexible configuration options.
  4. Answer 10k RPC requests per second for getAccountInfo on ordinary hardware.
  5. Store even very large (tens of GB) Solana programs.
  6. Recover from crashes or restarts without needing to re-build the cache.
  7. Do all of this on the cheapest FluxRPC plan (the 38$ a month Developer plan)!
A bandwidth-saving data stream feeds the accounts you pick into Lantern, which answers your app's RPC calls locally and only reaches out to an RPC on a cache miss

With the last release (0.0.13), we've finally completed all these features, and even added a couple more. For example, you can use Lantern to send transactions directly to the validator. It probably won't beat the people who can pay for optimized routes, but at least you save the extra hop to an RPC! It's also free (the best price), and has no rate limits (the best kind of rate limit).

Lantern is available on all FluxRPC plans, even the Free plan. If you sign up, we'll give you 10GB of free bandwidth to try it out.

Optimizations to Solana-Go

Our development relied on Solana-go heavily. Over the last couple of years though, we ended up writing our own code to handle the same tasks either faster, with less memory, or with fewer RPC requests. We ended up with a spec-complete Solana SDK for Go, built around performance and long-term API stability. We recently released it open-source!

You can find more details here. And the source code is here.

One neat thing we did is optimize the base58 codec heavily. The performance difference is quite large for longer base58 strings!

Under the Rug