4 Major Challenges When Using Nodes (And How to Overcome Them)

More developers than ever are using blockchain technology to build their applications. Since BullPay started in 2017, we’ve been obsessed with improving the experience of blockchain interaction. Blockchain technology has powerful benefits, but there are challenges with fetching blockchain data in a useful manner.

1. NODES WERE DESIGNED BY ENTHUSIASTS; ARE OPEN SOURCE.

This means:

1. Only few public resources are available — Compared to a well-known server provider like Microsoft SQL, a blockchain node actively lacks publicly available information.

If you use a node for anything more than a hobby or a small-scale wallet, be ready to get your hands dirty with development and to seek additional resources.

2. Nodes can crash and are unstable — Releases sometimes come with problems that cause instability. Keeping a node running can be a challenge and may require reverting to older versions and/or if open source, direct modification of the nodes code.

3. Updates can break usage — If you’re making requests to a node in some way, that way may change. It usually occurs through deprecation where the creators slowly phase out support. It can also be a sudden change.

In either scenario, a node upgrade may require you to go back and develop new calls to the node.

2. NODES WERE DESIGNED TO BE ECONOMICAL, NOT FOR ENTERPRISE.

While it is true that stability can be achieved for a normal running node. However, if you put that node to work, start hitting it, and pounding it with hundreds of request… it will inevitably become non-responsive.

How many requests can that node serve up before it destabilizes and crashes? Plan on testing it. Multi-threading requests to a node can be quite problematic. We found it easy to overwhelm a node with requests.

The solution was to blow out the data into a larger scalable database designed to handle billions of records, search them and provide massive result sets at incredible speeds.

The other route you could take is to spin up multiple nodes and attempt to load balance the requests among the nodes.

If you plan on scaling beyond a small project, plan on having to set up many nodes to handle large scale requests and design management software to monitor those nodes. If you deal with more than one blockchain, you’ll have to repeat the whole process over again and scale the number of nodes to match the need.

3. NODES ARE SLOW.

Mainly because of these two reasons:

1. Size — To keep nodes running smoothly on a mid to low range hardware, many nodes only commit data in intervals of time or quantity.

For example: We have a node that by late October 2018, on restart would go back to April of the same year. Only ~80% of the entire chain was stored. When starting the node on high end hardware, it still took ~15 minutes to catch back up. By November the data was committed and, on a restart, has almost all recent blocks. As time progresses, the time it takes to restart this node will increase until the next data commit.

In sum, it’s not something you can turn off and expect it to come back on instantly.

2. Nodes were not designed to be fast searchable data repositories — They were designed to keep the bar to entry (running a node) as low as possible. That means data is stored as compacted as it can be. There is no way to streamline searching of the data as it needs to be translated from stored hashes before being served back.

The downside to this is that getting anything other than a small amount of data from a node can take quite a while.

4. EACH BLOCKCHAIN NETWORK OF NODES ARE UNIQUE.

Each blockchain has its own characteristics. Depending on the origin of the node, the setup can be remarkably different, i.e. a Geth node for Ethereum is different than a Bitcoin node.

Nodes are set up and presented with a number of options. For instance, in processing a new block, some nodes provide more data at the expense of disk space or performance.

Developers creating their own web tools interacting with blockchains need to ask themselves the following questions:

Do you need to be sure the integrity of each block?
Will you use Proof of Work checks?
Consensus checks?
What’s the impact on performance if you do?
What problems could arise in your solution if you don’t?

There are lots of questions, research, testing and decisions to be made with each blockchain you implement.

Not to mention many odd things that can happen with each node.

For example: Dash is sometimes spammed by “bad blocks” that do not pass Proof of Work (PoW) or Consensus checks. Even though the blockchain can resolve this within a few blocks, those blocks working at the tip can have a wild experience — You will be forced to figure out the right way to deal with it.

Longhorn makes it easier for blockchain developers to implement blockchain API to develop their own solutions.

Longhorn by BullPay makes interacting with blockchains extremely simple for developers that want a robust way to send and get blockchain transactions and accurately show a wallet’s total balance. In the coming months, we hope to support additional blockchains that developers want to make more scalable blockchain-related projects and businesses.

We can work with wallets and companies that want a way to make more secure payments to businesses globally.

Ready to get started? Developers, sign up for a Longhorn account to try our blockchain API. Remember, you can make up to 5,000 calls per day for free.

Have questions or feedback about Longhorn? Send us a question or an issue report to our Email Support Team.