This article was originally published by AI Frontier.
Former Airbnb engineer: How to become a blockchain developer from scratch
Planning Editor | Vincent
Author | Haseeb Qureshi
Compile | Debra
Edit | Emily
“Cryptocurrencies, ICOs, money created by the Internet — all of this stuff is so damn exciting that you, the developer, can’t sit still and want to embrace this crazy party? But where should you start?
I’m glad you share my interest in this field. However, MY guess is that you may find yourself in a confusing situation at the moment, as there is no clear way to learn about blockchain in this era of rapid development.
Since I left Airbnb to work full-time on blockchain, a lot of people have asked me how I got into the field. This is my “definitive” but incomplete guide to getting you started in blockchain development.
This guide will be divided into ten parts:
- Why learn blockchain development?
- A prerequisite for
- The theoretical basis of Bitcoin
- Build your own blockchain
- Ethereum and smart contract programming
- Smart Contract Security
- Disengage training wheel
- Create your own project
- Explore the blockchain community
- Get a job
Why learn blockchain development?
Before I answer that question, let me give you a word of caution: Blockchain is an overrated space right now. These prices are unsustainable and will eventually collapse. All this has happened before, and could happen again. But if you work in this field for a long time, you have to learn to rule out price. In the words of Emin Gun Sirer, “Price is the least interesting part of cryptocurrency.” Blockchain is a very important technology and will undoubtedly change the world.
If you’re not sure, I can’t tell you if you should be in this circle. But there were five things that convinced me to get into it in the first place:
It’s still early.
Bitcoin was born a decade ago, but has only caught on in the last decade, especially with the launch of Ethereum in 2015. Most of the new companies and ideas in this space are based on Ethereum, which is still very immature.
So, even if you start learning now, you can become a world-class expert in a few years. Not many people do this, so it’s not hard to catch up with the team. It’s like someone who started deep learning in the late 20th century.
The field does not yet have a well-developed talent training channel.
Most of the best and brightest focus on machine learning, web programming or game development. Sure, blockchain is becoming more and more attractive in public discourse, but it’s still a weird and disruptive topic that can advance your career.
In its early days, blockchain was the exclusive province of cryptophiles, paranoids, and strangers, until recently. As a curious and open-minded developer, I’m sure you’ll bring a lot of value to the field.
Much of the innovation takes place outside academia
As far as I know, Satoshi Nakamoto is not an academic. No university or institution offers a coherent blockchain course. Much of the innovation in this area has been led by hobbyists, entrepreneurs and independent researchers. Almost everything you need to know can be found in white papers, blog posts, public sources, and open source software. All you have to do is roll up your sleeves and get to work!
The demand for talent far exceeds the supply.
There aren’t enough developers in this area, and they can’t be trained quickly. Every company is competing to hire blockchain talent, and project staff should be the ones who feel the scarcity most personally. Many top companies can’t hold on to talent because the pay isn’t attractive enough because they have too many options. It’s easy to find a job if you master some skills.
Cryptocurrencies are really cool.
What other job in the world can create things like password protection and decentralized currency that only come out of science fiction movies? It’s like the Wild West, with both good and bad consequences. There will be more transparency and better regulation. But there’s no question that cryptocurrencies are one of the most innovative areas you can work on right now.
The key to success, Naval Ravikant said in a recent interview, is to give society something it wants, but can’t get on its own. You can’t learn these things in school, otherwise they would have been available all over the world.
So you should create something that no one else can do. At the moment, blockchain is something new, something the world knows very little about, and if you succeed in building the future of decentralized technology, the world will treat you well.
A prerequisite for
I recommend that you know the basics before you dive into this area. Blockchain is built on decades of research in computer science, cryptography and economics. Satoshi Nakamoto is a renegade, but even he knows the history of blockchain well. To understand how blockchains work, you need to understand their building blocks — what else came before blockchains, and why they don’t work.
Here are the prerequisites we need to be familiar with, in order of importance.
Please note that these links are just a starting point and you will need to do further research on these topics.
Computer science
The data structure
You need to familiarize yourself with the characteristics and complexity guarantees of major data structures: linked lists, binary search trees, hash maps, and pictures (especially directed acyclic graphs highlighted in blockchains). It helps build blockchains from scratch to better understand how they work and their properties.
encryption
Cryptography is the namesake and foundation of cryptocurrencies. All cryptocurrencies use public/private key encryption as the basis for identity and authentication. I recommend you study RSA (it’s easy to learn, you don’t need a very good math background) and then ECDSA. Elliptic curve cryptography requires more abstract math, and while you don’t need to understand all the details, be aware that this is the cryptography used in most cryptocurrencies, including Bitcoin.
Another important cryptographic primitive is the Cryptographic Hash function. These can be used to build Commitment schemes, which are the building blocks of the Merkle tree. The Merkle tree makes the Merkle proofs one of the key optimizations for blockchain scalability.
Distributed system
Here are a few good textbooks on distributed systems, but they are broad and difficult to learn. Distributed systems are critical to blockchain reasoning, so you must have some basic knowledge of this before you start programming blockchain.
When you work on more than one machine, you have to start reasoning about consistency and consensus. You need to understand the difference between linearizable and eventual consistency models, the guarantees of fault-tolerant consistency algorithms such as Paxos and RAFT, and the reasoning time challenges in distributed systems, and the trade-offs between security and survival.
With this background, you can understand the difficulties of Byzantine fault-tolerant consensus, the basic security requirement of public blockchains. You also need to know about PBFT, one of the first scalable algorithms to provide a Byzantine fault-tolerant consensus. PBFT is the basis for many non-proof-blockchain consensus algorithms. Again, you don’t need to know the specifics of PBFT correctness, just the general idea and security guarantees.
It’s also useful to understand the traditional approach to distributed databases (core, because blockchain is a database after all). You also need to know about sharding (e.g. through Consistent hashing), leader-follower replication, and quorum based commits, as well as distributed hash tables. Such as Chord or Kademlia.
network
The distribution of blockchain mainly comes from its peer-to-peer network topology. Thus, blockchain is a direct outgrowth of the P2P networks of the past.
To understand the blockchain communication model, you need to understand the basics of computer networking: this means you need to know TCP vs UDP, packet models, IP packets, and how Internet routing works.
Public blockchains usually use the Flooding gossip protocol to spread messages. It’s helpful to know the history of P2P network design from Napster to Gnutella, BitTorrent and Tor. Blockchain has its place, but draws on the design lessons of these networks.
economics
Cryptocurrencies are inherently interdisciplinary, which is part of what makes them fascinating. In addition to computer science, cryptography and networks, they are inextricably linked to economics. Cryptocurrencies can gain many security attributes from their economic structure, often referred to as cryptoeconomics. Economics is therefore crucial to understanding cryptocurrencies.
Game theory
Game theory plays the most important role in cryptocurrency economics, the study of returns and incentives between multiple agents. You don’t need to know much about this, but you do need to know the basic tools of game theory analysis and how to use them to analyze incentives in both primary and iterative games.
Two key concepts need to be known about game theory, namely Nash Equilibrium and Schelling points, because they play a prominent role in implicit economic analysis.
macroeconomics
Cryptocurrencies are not just protocols, they are also a form of money. It is therefore subject to macroeconomic law (if you can call it law). Cryptocurrencies are influenced by different monetary policies and respond predictably to inflation and deflation. You should understand these processes and how they affect spending, saving, and so on.
Another valuable economic concept is the velocity of money, because it affects its value.
microeconomics
Cryptocurrencies are also tied to the market, which requires understanding microeconomics. You need to be very intuitive about supply and demand curves and have the ability to reason about competition and opportunity costs (often used in cryptocurrency mining). Auction theory is particularly important for many coin allocation and implicit economic systems.
Now that you’ve mastered the basics, get started on blockchain development.
The theoretical basis of Bitcoin
In October 2008, Satoshi Nakamoto published a white paper on protocols for decentralizing digital money. He called the protocol Bitcoin.
Before you can understand the big idea behind blockchain, you must start with Bitcoin and understand Satoshi’s original intention.
To start, I suggest you get acquainted with proof-of-work and the Fork Choice rule (also known as the Satoshi Consensus), starting here:
Video link:
V.qq.com/x/search/?q…
I suggest you watch a lot of video commentary and let this idea take root in your mind:
Video Link 2:
www.youtube.com/watch?v=bBC…
With this intuitive understanding of the blockchain in mind, this article will take a deeper, end-to-end look at the key components of Bitcoin.
Build your own blockchain
Now it’s time to build your own proof-of-work blockchain. Don’t worry, it’s easier than it sounds, and here are some great resources.
First, I made a video lecture on how to do this in Ruby (even if you’re not a Ruby programmer, I encourage you to watch it) :
Resources: github.com/Haseeb-Qure…
Here you’ll also find other blockchain apps written in a variety of programming languages. Continue to build your model.
By the time you’ve done this, you should have a good grasp of how to create a simple payment application on the blockchain (Bitcoin). You should also have enough background knowledge by now to be able to read and understand the original Bitcoin white paper.
To understand the economics and mechanics of bitcoin mining, I recommend watching Princeton’s Bitcoin Mining Lecture on Bitcoin and cryptocurrencies.
By now, you should have reached a point where you know the Bitcoin headers and know what each component means. You can also explore the original Bitcoin transactions using the Bitcoin Block Browser.
Now is a good time to study the history of bitcoin and cryptocurrencies. The following UC Berkeley video gives a good overview.
Video link: Youtu.be /apYieuvnUaE
Some additional resources:
- The academic pioneer of Bitcoin
- Bitcoin mechanics: UTXOs and Bitcoin script (Bitcoin script is not very important, just know roughly what it can do)
- The Bitcoin Forks Guide
- Soft fork and miner signal
- Double whammy, 51% attack and selfish mining
- Replay attacks
- Bitcoin scalability issues are the source of much of the controversy in the Bitcoin ecosystem. You know why
- Bitcoiners have a lot of arguments about the size of the block.
- Segregated witnesses, known as SegWit, are not required to be known, but will appear frequently.
- The Lightning Network is one of the more important scaling solutions for Bitcoin, and is spreading to other blockchains as well
- Bitcoin complete node, Bitcoin fee statistics, charts, charts and other charts
- Bitcoin Energy Consumption Index (at the time of publication, bitcoin mining used as much energy as the entire country of Peru)
- Gwern makes an insightful argument about the nasties of bitcoin
- If you want to take a closer look, check out Jameson Lopp’s resource on Bitcoin resources.
Ethereum and smart contract programming
Now that you’ve built your own blockchain and know the dynamics of Bitcoin, it’s time to look into Ethereum.
Understanding blockchains and workflows demonstrate how distributed Byzantine fault-tolerant consensus can be implemented within peer-to-peer networks. But payment networks are just one of the applications you can apply blockchain to. In 2013, Ethereum founder Vitalik Buterin asked: What happens when blockchain is applied to distributed computers?
On Ethereum, you pay miners to run their own programs on this distributed virtual machine. This means that you can perform arbitrary calculations using the Turing full programming language (as opposed to bitcoin scripts). Obviously, this includes all payment processes, so Ethereum Bitcoin is more powerful and innovated further.
After that comes the smart contract, the name of the program that runs on such a virtual machine. Smart contracts can interact directly with the blockchain’s digital currency based on the execution of the program. In other words, you can create financial contracts that automatically execute. It’s a crazy idea, and with this programming model, you can do almost anything in science fiction.
Ethereum has set off a wave of ICO and blockchain development. It is the second largest cryptocurrency after Bitcoin and has 10 times as many developers as the third largest cryptocurrency. It has the strongest development team, the most mature tools, most ICOs and projects related to it, and the most promising in the industry. If you’re doing blockchain development, chances are you’ll be writing code for Ethereum smart contracts. (Even if you don’t want to, it doesn’t hurt to check out the trends in the field.)
First, more on high-level Ethereum:
Video link:
www.youtube.com/watch?v=mCz…
The ideas behind Ethereum have also spawned a wave of innovation in hidden economics. You should get in touch with DAO ideas.
Well, the fantasy is over, and let’s dive into the technology behind it.
Preethi Kasireddy’s overview of the Ethereum Yellow Book and internal architecture. Ethereum uses an account model rather than Bitcoin’s UTXO model, and you’ll soon see why this makes writing smart contracts easier.
As with any technology, the best way to learn about Ethereum is to create small projects.
Ethereum’s main programming language is Solidity, which is a statically typed javascript-esque language with a lot of shortcomings and design issues to be honest. More powerful languages like Viper could replace it when it’s launched, but for now, Solidity is the universal language for smart contract programming. This is basically JavaScript for Ethereum, so you need to learn it (and its pitfalls).
For your first exposure to Solidity development, I recommend that you read through all the CryptoZombies tutorials. These are high-quality Codecademy-esque tutorials to learn the basics of Solidity programming.
Now that your appetite is up, it’s time to develop your own.
The task of getting started with Ethereum is to build an ERC-20 compatible token. I recommend this guide as a guide for completing your tasks.
Remix is a Solidity editor and compiler in the browser that is basically the training wheel for Ethereum development, so I recommend doing all the other exercises in Remix. Setting up a local blockchain to learn about ethereum tools is also worth a try. This tutorial provides a good introduction to an end-to-end blockchain stack and explains the stacks.
Next, I suggest you create a voting system called Todo for Ethereum. Karl Floersch’s tutorial explains how to set up a secure submission display voting system.
Good, now it’s time for the midterm exam: Create a safe coin-op game where two players can play a coin-toss game safely. There is no tutorial this time, so make it yourself. Consider possible attacks during this process, such as how can the player cheat? Can you make sure they’re being honest? Here are some hints.
The security of smart contracts
Security is an absolute necessity for blockchain development. Smart contracts have been threatened by catastrophic hacks of daOs, Parity Wallets, and Parity Wallet Hack 2 (and now their own T-shirts). If you are going to write a production smart contract, you must thoroughly analyze these three hacks.
In reality, however, smart contracts are hardly perfect. While improvements in programming toolchains have made it harder for hackers to carry out attacks, programmers will inevitably make mistakes. Smart contract programming can also make many subtle mistakes, such as preemptive trading or secure random generation.
As a smart contract developer, you must put security first, and all code that handles large amounts of money flows should be run through a static analyzer like Oyente or Securify, thoroughly tested, and then audited by an experienced smart contract auditor. For more reliable results, you can also use prevalidation components such as the OpenZeppelin open source contract.
For security purposes, I recommend using OpenZeppelin’s Ethernaut, where you can find vulnerabilities against smart contract hackers, which is a great way to prevent attacks.
Phil Daian created a famous smart Contract hacking challenge called Hack This Contract.
After that, I strongly recommend reading Smart Contract Best Practices, written by ConsenSys. Expect to revisit this document many times in your smart contract programming career. References recommended by security experts are also worth further reading.
Disengage training wheel
If you’ve done this, you should now be ready to go through Remix and start developing the stack in earnest using Solidity.
Most developers recommend using VSCode or Atom as text editors because their Solidity plug-ins are quite sophisticated. To interact with local blockchains, you need to use Ganache (formerly known as TestRPC), as well as the Truffle framework for testing (JS-BASED) and configuring build pipelines.
Now is a good time to explore IPFS as a much cheaper distributed repository than Ethereum. Here’s a brief note from founder Juan Benet:
youtu.be/skMTdSEaCtA
Most developers recommend Infura for interacting with Ethereum and full IPFS nodes. On the Ethernet network, Etherscan and ETH gas stations provide useful real-time statistics.
Once the Web3 stack is set up, try deploying an end-to-end Dapp (distributed application). This tutorial is a complete stack overview using Node and Postgres as the back end, in addition to demonstrating how to use IPFS as the persistence layer to create a fully decentralized application.
Create your own project
By now, you should be familiar with most of the technology, and all that remains is to start building projects and delve into the blockchain community.
First, start creating your own projects. If you have some great idea that excites you, go for it, and invite friends to attack it! If you don’t have a good idea yet, there are plenty of high-quality open source projects to get started on. OpenZeppelin is a good choice for creating smart contracts.
In addition, I suggest you pick a development project you’re passionate about, go into their Slack or Rocketchat, tell them you’re making a contribution, ask them to assign you small tasks, or find unresolved issues on their Github.
Note that while I have focused on protocol and smart contract development, blockchain companies need Web developers to build their core functionality. These roles often need to interact with the blockchain, so you should have some understanding of the blockchain model. But for many engineers at blockchain startups, most of their work is building a Python web server, or designing a React front end, interacting with blockchain may be only a small part of the job. You don’t have to focus on smart contract development, because it’s really just part of blockchain engineering.
In addition to open source contributions, there are many blockchain hacking contests popping up. Most projects have a free public Slack that you can join, as well as a very active Ethereum Gitter community. As you get deeper into this area, you will eventually find like-minded partners in Telegram, Gitter and other places to learn and progress together.
Blockchain Community
The best way to really get to know the world of blockchain is to immerse yourself, read and listen to the smartest people, especially what they’ve written in the past. This has always been my strategy when trying to break into a new field, and it has paid off handsomely for me.
While there is a lot of good blockchain content, there is also a lot of nonsense. Here are my recommendations for learning communities.
The media
I recommend three great podcasts, Software Engineering Daily Blockchain Interviews, lots of introductions and tutorials on technology and cryptocurrency. Epicenter and Unchained, you might want to go back and listen to some of the programs that have already been on the air. Another interesting tech podcast is knock.
Also have a couple of great Youtube channels (despite a lot of crap on Youtube) like Ethereum Foundation and watch the Devcon3 presentation. Berkeley’s blockchain course records many of their lectures, most of which are great technology overviews. Other technology channels include Decypher Media and Jackson Palmer.
Read online
There are two places where blockchain enthusiasts usually air their opinions: Reddit and Twitter. Most of the users on Reddit are of low quality and mostly crap. Content on R /Ethereum is consistently good quality (and there are some well-informed users for specific cryptocurrencies). However, most of these people are investors, so don’t pay too much attention to what they say. Far from the users involved, the bitcoin community is notorious and shuns.
Twitter is even more mixed. In any case, most blockchain people live in Twitter. In my experience, there are five types of blockchain people here: developers, entrepreneurs, journalists, businesspeople, and “thought leaders.”
For these people, I suggest you steer clear of “thought leaders”. Entrepreneurs don’t matter, journalists usually focus on the big news, and unless you’re a business person, it’s just a distraction for developers.
The most important thing to focus on is the real developers. At present, they are the most important group, and they are driving the development of technology.
Here are some examples of specific types:
developers
- Vitalik Buterin, Ethereum
- Zooko Wilcox, ZCash
- Nick Szabo, inventor of smart contracts
- Vlad Zamfir, Ethereum
- Marco Santori, Cooley LLP
- Riccardo fluffypony Spagni, Monero
- Matt Liston, Gnosis
entrepreneurs
- Balaji Srinivasan, Earn.com
- Erik Voorhees, transform
investors
- Naval Ravikant, MetaStable
- Ari Paul, Blocktower Capital
- Linda Xie, Scalar Capital
- Chris Burniske, Placeholder
The reporter
- Tuur Demeester, Adamant Research
- Laura Shin, Forbes (you can follow me, too, though I’m definitely not on this list.)
I suggest you follow Twitter and Reddit as little as possible, and just follow a few news bulletins like Inside Bitcoin, Token Economy, And Week in Ethereum to stay up to date.
Beyond that, you don’t need to monitor the latest news in real time, but focus on project creation and learning.
Here are some good tech blogs:
- Vitalik Buterin, great blockchain and crypto-economic analysis (Read all his previous blog posts, Vitalik is widely considered the thinker for a generation of blockchain people)
- Hacking,Distributed, blockchain security analysis by Cornell University researchers
- Unenumerated, Nick Szabo’s blog post on the role of cryptocurrencies in society
- Money Stuff: Bloomberg Syndication, Matt Levine’s in-depth analysis of market, financial and blockchain news
- Vlad Zamfir is moderate and cautious on state-owned and public blockchains
- Chris Burniske has written a series of high-quality blog posts on how to value crypto assets
- Jameson Lopp has written a great series of technical articles from the perspective of a blockchain ecosystem software engineer
- Tim Swanson’s Great Wall of Numbers is a sobering and coherent deconstruction of blockchain mania, especially in the corporate world (you can also read my blog, again, though I’m still not on this list).
Books and Lessons
If you want to learn blockchain in a more systematic way, you need to read some high-quality books and courses.
The best textbook on blockchain is Bitcoin and Cryptocurrency Technologies (includePrinceton Coursera). Other books in this area I only recommend Mastering Bitcoin by Andreas Antonopoulos, co-authored with Ethereum founder Gavin Wood and published by O’Reilly. For non-technical books, I recommend Nathaniel Popper’s Digital Gold. Almost everything worth reading is on blogs, not books, because the field is growing so fast that the most important people rarely have time to write books, and books are often out of date by the time they are released.
You can also learn systematically through some high quality courses. Here are links to some Princeton Coursera courses (also available on Youtube) and some lectures from UC Berkeley. Consensys Academy’s courses are also good for those who want to get into smart contract development.
I also teach a 4-week cryptocurrency workshop for software developers at the Bradfield School of Computer Science in San Francisco. If you’re a software engineer in San Francisco and want to learn more about the theory and practice behind cryptocurrencies, take a listen.
Looking for a job
As I said before, blockchain startups are hiring like crazy. If you do, or even half of, what I’m talking about, you’ve already made a difference in this area. AngelList does a great job on how to find work in the cryptspace.
Here are a few good blockchain-related work posts:
- AngelList encryption started
- BlockchainJobz
- Ethereum Jobs
- Be in Crypto
- Blockchain Working Committee
- Crypto Jobs List
- Google (Blockchain search query)
- ConsenSys Jobs (who owns a number of Ethereum startup studios)
I know of some particularly promising blockchain startups that are hiring developers:
- 0X
- Dharma Laboratory
- Civic
There are also some big companies out there looking for developers:
- Coinbase, the Google of cryptography, has been on a hiring spree
- Stellar and Ripple, if you want to be business-friendly cryptocurrencies, they’re good choices
- Square has already integrated some blockchains, but it’s not known if they’re hiring externally
- IBM, Visa or JP Morgan
But in my opinion, it is best to choose a company because you are interested in a project and can directly participate in its development. Most blockchain teams are willing to hire remotely for the right talent. Many developers can use Twitter, Github, or public Slack channels to show off their technical prowess and initiative, and they can make an impression.
This is my advice to you, after doing this, I believe you will go much further than I did.
Rabbit wormhole
What I’m showing you is just the beginning. Cryptocurrencies are still in their infancy, and I really believe this is the fastest growing area to invest in. I believe this guide will be out of date within a year, and you’ll need to keep exploring, keep improving, keep learning as you enter the field.
For more content, you can follow AI Front, ID: AI-front, reply “AI”, “TF”, “big Data” to get AI Front series PDF mini-book and skill Map.