- 10 Years of open-source Visualization
- Originally written by Mike Bostock
- The Nuggets translation Project
- Permanent link to this article: github.com/xitu/gold-m…
- Translator: momodiy
- Proofread: PassionPenguin, Chorer, W-qing
Ten years of open source visualization
See what I learned from d3.js…
To mark the 10th anniversary of the release of D3 1.0, I thought I’d reflect on the lessons learned. I’m not trying to be overly comprehensive or serious, but this is just a glimpse of the next decade, but I hope there are one or two insights that will interest you.
1. The most important thing in building a wheel is teaching others to use it
It’s easy to forget how much you’re simplifying when you’re building a wheel, which means how much you’re making assumptions about what the user or reader already knows and context. Your tool may be familiar to you, but it will be completely alien to others. If your goal is to get someone else to use the damn thing, that means you’re not just building the wheel for yourself (which is a perfectly valid reason), you have to help someone else use it! It doesn’t matter what you might do or plan to do, what matters is whether users can successfully use it in practice.
To maximize your impact, your core strategy should be teaching. This includes documentation, tutorials, examples, videos, tweets, and more. One-on-one teaching or seminars can be a great way to surface your knowledge and find common ground with your audience, and you can learn from their work and learn how to teach more effectively (see section 2). But as your student body grows, creating a series of tutorials is the only way you can scale without spending personal time. Your tutorials are your passive income if needed. A day of studio training can help 100 people, and a tutorial can help tens of thousands or more. I’m also often inspired (and occasionally envious) by the great teaching styles of Rich Harris and Dan Abramov.
Of all the types of documentation, examples seem to be the most effective. Observables and its predecessor, Bl.ocks.org, both speed up learning by letting people create and use samples. Examples can be inspirational by showing how they might be used, and being able to demonstrate specific techniques is a cornerstone to help people get started. Given that people are good at pattern matching and extrapolating from observations, examples are naturally the best material to learn from.
Examples are so powerful that they compensate for a difficult API and promote a reliance on copy and paste. D3 is successful not because it’s a great tool, but because it comes with lots of examples, isn’t it? 🤷 EVEN though I’m the author of D3, I can’t remember how to use D3. stack either, and I have to copy and paste the code examples for the stack section like everyone else. Examples may blind people to the tool’s flaws and help them succeed, but more important is the pursuit of proficiency — internalizing tool usage and learning to build from scratch. It can also be a mistake to start with examples instead of data for visualization (see section 5). Examples also have limitations: examples that teach specific skills may not be broadly representative of usage in real projects.
2. Support is a powerful research tool
The only way to understand the understanding gap between you and the tool users is to focus on their pain points and communicate with them. In this way, some obvious flaws will surface almost unbelievably quickly. Answering questions on Stack Overflow (or GitHub, Twitter, Slack, etc.) isn’t altruistic; it’s a learning opportunity to understand your users’ pain points and hear their point of view. Each question is an opportunity to help others, and the questions can also be used as tutorials or examples to help others avoid the same problem. This potential advantage can also make successful tools more successful: maintainers can improve projects or refine documents with the input and criticism of countless users. You know, no idea comes out of nowhere.
But there is a limitation. You can’t help everyone on your own. Also, your tools are only a small part of their lives, so you can’t satisfy everyone. So instead of making their goals your goals, you should focus on learning and broadening your horizons. If you can’t get any more constructive help, stop and don’t get discouraged. I’m not a big deal about the way GitHub and other platforms have enabled message alerts by default, because it suggests unreasonable expectations that unpaid maintainers must immediately respond politely and with substance to all requests for help. Yes, I can close issue, but as members of the community, if we are serious enough about addressing maintainer burnout, we need to rethink our behavior.
3. Be careful: Interaction, animation, or other technical highlights come at a cost
These things do have to be presented, but please listen to me carefully. For users who don’t often encounter animations and interactive effects, they can be very impressive. Knowing this, you might add these features to your visualizations without realizing the downside: they add complexity to the project and hide the actual value of the information behind the interactive controls. Worse, because of the difficulty of implementing these features, they can distract you from the more important but boring real tasks.
This is not a moral judgment, and I’m not accusing you of doing something bad. In fact, I just feel guilty about myself. However, there are drawbacks to this. The first thing we should focus on is the static form, which for some readers may be the only thing they see. Don’t let technical issues, such as Web development, get too much of your attention. And don’t worry about a simple visualization either: what matters is what we can learn, not how fancy it looks. So where should we use interaction effects? The main purpose is to explore visualization (see section 4). Gregor’s In Defense of Interactive Graphics is a good answer.
4. Visualization is a process from exploration to interpretation
Not all visualizations serve the same purpose, and visualizations you create yourself can show data in a new light. An explanatory chart, by contrast, conveys to the viewer what is already known. So find your niche when designing.
The main purpose of visual exploration is to increase speed: How fast can you build a view that answers your question? When your target audience already has some background, you can omit things, but you must provide diagrams to explain the context. A good visualization should know what it wants to say and be able to say it correctly. Diagrams can also contain illustrative elements, such as allowing readers to see what they want in the data, but ideally these elements should not interfere with the main information. It should be the author’s job not to make the reader worry too much about a clear focus.
5. Generally speaking, 80% of visualization is processing data
Visualizations are the end product of an analysis, a visual representation of data that can be viewed, shared, and understood by professionals and non-professionals alike, and thus can be overpraised. To achieve visualization, you must first go through the steps of finding data, cleaning it, transforming it, linking it, modeling it, etc. Data processing is sometimes dismissed as a cleaner, but nonetheless represents a critical step in understanding the nature of data. (See Leigh Dodds’ article on this topic.) If the data structure is correct, the process of applying visual coding can be relatively simple (assuming you’ve seen section 3). Therefore, the D3 modules I use most often are D3-Array and D3-DSV. I’m glad to see new Javascript tools like Tidy. Js and Arquero for manipulating data. Hadley Wickham’s examples of collating data are valuable.
6. Don’t limit yourself to a specific visual presentation until you see the data structure
A certain visual form (such as a bar chart or tree chart) is not necessarily good or bad, but whether it is appropriate for your data and the specific question you are trying to answer is questionable. There’s only one way to know if the graph you’re using is accurate, and that’s to see what it tells you: you have to look at it after you combine the data, and don’t start with a particular presentation, but think about what kind of problem your data is trying to solve.
If your data is hard to determine what type of example it corresponds to (see section 1), try a specific type of chart first to make sure it works. The less effort it takes to draw a visual, the more variations you can try, and the better the result, which is why Observables help you get data quickly, like replacing files with a single click or letting you edit code without having to fork. But if your data structure isn’t compatible, you may still need a lot of work (see item 5). Also, D3 is primarily designed for custom explanatory graphics, as things like Vega-Lite for exploring graphics require more work. If you want to reduce the viscosity of a particular symbol’s cognitive range, you can quickly try many forms and see what works. We are making it easier to draw visualizations from Observables and hope to share them soon. Stay tuned.
7. 10% of the code causes 90% of the bugs
I made these numbers up, but I feel the conclusion is correct: some code is simply more prone to errors than others. Not because the code is of low quality, but because it is trying to do something that is inherently difficult or uncertain. The most buggy things in D3 are the interactive behaviors, such as D3-Zoom, D3-Drag, and D3-Brush. It is difficult to reason by controlling variables, let alone test for all possible asynchronous events. At the same time, the interaction is also uncertain. Are you clicking, dragging, panning, or double-clicking? What’s more difficult is that the browser can change, as passive event listening simply destroys the page.
The above shows that you can reduce some problems by being careful about what you solve and what you don’t. For example, if you build pages using Observable input and data streams instead of low-level event listeners, you’ll have less to worry about.
8. There are always people on the Internet who make you sad
No matter how good your job is, if you put yourself out there on the Internet, there will be people who will say hurtful things and make you feel bad. This is usually not intentional, but it doesn’t matter. I am proud of D3, but I have also collected some negative tweets that others have shared. This is how I deal with it (collecting negative tweets), please don’t judge me. (Oh, I’m not sharing this list.)
I have no ill will towards those who get frustrated using tools, and I fully understand how helpless a tool can be — you only need to design a small form, but you have to learn millions of other things. The problem with the Internet is that the design of a tool can be so arbitrary and slow. In the past you would have complained to a colleague or friend about how bad D3 was and maybe they would have helped you out and knew you were venting. Because of the Internet I hear these complaints too, I don’t know you, and they frustrate me. If I developed tools to see the joy of successful users, and now I feel nothing but pain and frustration, why would I continue to develop? Why should I be belittled and denied when all I want to do is watch YouTube?
So if you find yourself complaining on the Internet, please consider the real impact of your words. Does this only lead to developer slackness and frustration? Or worse, shut down open source projects? Or better yet, you can contribute something to make the tool better, such as pull requests or document support. That, after all, is the beauty of open source.
9. Don’t be alone
To avoid pinning your happiness on the Internet (see section 8), you must develop a small, stable organization that you respect — in other words, find a team or organization that can provide validation, feedback, support, and guidance. Everyone knows this, except me — yes, Mike, it would be nice to have friends. Still, I think it’s important to point out that most of today’s communication takes place on the remote Internet. When I build a remote shared collaboration space using Observable, I’m happy, not to show off (like on Twitter), but to be able to collaborate with friends in real time to post insights and solve problems.
Have a good time
At first I wrote “relax”, but it felt a bit fake. I’m probably the last person to relax. I want to be able to take it easy. It’s a personal thing for me, and I try to reflect on the parts of my work life THAT I enjoy the most and spend more time doing them. It sounds simple and cliched, but it’s not easy to do. If you know what you love to do, then (in case) you fail (hypothetically, not really), you’ll have fewer regrets. Paradoxically, a goal that takes you away from your actual purpose may help you succeed because you’re more likely to stick to it. It’s much easier to stick to doing what you love (work), like I love building tools that I can’t predict will work, but I love solving puzzles and abstracting, and I love seeing people use my work. On the other hand, while I know the benefits of public speaking and how great I feel about myself afterward, I find public speaking to be anxie-inducing. So if you’re a little disappointed by my lack of public speaking, I hope it’s only because I’m working on something new. 😅
If you find any mistakes in your translation or other areas that need to be improved, you are welcome to the Nuggets Translation Program to revise and PR your translation, and you can also get the corresponding reward points. The permanent link to this article at the beginning of this article is the MarkDown link to this article on GitHub.
The Nuggets Translation Project is a community that translates quality Internet technical articles from English sharing articles on nuggets. The content covers Android, iOS, front-end, back-end, blockchain, products, design, artificial intelligence and other fields. If you want to see more high-quality translation, please continue to pay attention to the Translation plan of Digging Gold, the official Weibo, Zhihu column.