This topic, a lot of companies will be out, because now there are too many picture applications, just image filter, to support a lot of companies. But here we look from an overall point of view, by default, we are facing the design of a similar 500px or petal network as a picture application website, around the user experience, performance optimization, network deployment and other big problems, we will have a clearer understanding of the system design.
Without further ado, let’s begin!
First of all, we need to make clear the portrait of the target customer group. Any design we do is for specific product scenarios. If the cost is too high but the audience is too low, it may not be able to meet the company’s later development.
For example, what do you think about these needs?
- Outside the Intranet web
- traffic
- Image quality
- Account privileges
- The social network
- Community types
- , etc.
System design is an open question, through continuous questioning to explore the subject’s knowledge, psychological quality, thinking ability and so on. Therefore, the following mentioned some content, just the author of the scope of knowledge and thinking, perhaps you have more to consider, the interview should be bold answer!
We are going to break this problem down into several sub-problems to solve. Of course, each sub-problem is not completely separate, but you should stand on a high position to control all the problems as you read the passage below.
- Requirements for entry
- Architecture design
- The front end design
- Network design
- The back-end design
- Database design
- Engineering problem
- After-sales service & user experience
1. Demand entry
Clear requirements are the primary task of software engineering, and an unclear requirement will lead to repeated development of the project, and eventually the system will age and die. Since our goal is to design an image app. At least these points can be analyzed:
- Customer scope. It is important to be clear that the development modes of public and Intranet applications are very different.
- Customer base. Including customer numbers, ages, preferences, geographic locations and more
- Application type. Mobile applications, web applications, desktop applications should be clear, or to support what types.
- Concurrent requests. Support multiplayer online? Estimate the concurrent traffic volume.
- App experience. What kind of user experience needs to be supported, for example, 500px targeted at high-quality users will have different user experience, which also involves the function of authority stratification and late charging.
- Basic functions. For example, picture applications generally support upload and download, picture classification and classification, account login and logout, multi-level user rights management, social functions and so on.
For most developers, they don’t care how the requirements are recorded, which creates a lot of problems because there are so many deviations in the paraphrasing process, like the polarization of light, that the product may not be what it was originally intended to be. So, again, it’s important for developers to identify requirements, and if necessary, go to the front line and talk to the customer yourself.
2. Architecture design
Obviously, once the need was identified, all that was left was to start building the wheel. But we shouldn’t be in such a hurry. Often 60% to 70% of the work of implementing a piece of software should be in the architectural design. Why? Architecture is like the foundation of a software project. It’s hard to imagine changing the foundation after the house has been built. It would be a disaster.
Usually, you need to have a meeting with all the engineers after you input the requirements, and you need to have at least one design
- Front desk architecture design, user – oriented
- Background architecture design, perfect service
- Network architecture design, deployment and access solutions
- Design database architecture and persist data
- Operational plan
At each level of subproblems, there are countless solutions, and we need to make some choices, and choose the best solution that meets our needs.
summary
Again, it’s important to be clear about the above two points. As a systems engineer, you don’t open your IDE every time you get a problem. Be a listener and a thinker, and try to find the best solution.
3. Front-end design
Finally, the concrete software implementation plan. When it comes to front-end development, there are so many technical solutions to choose from. Note that the choice of a solution must be based on the actual skills of your team. As the front-end development has a certain versatility, we can be divided into two perspectives to look at the picture application of the front-end design.
General plan
- Basic development language, framework selection
- Determination of packaging scheme
- State management: Basic data is generally in-memory and can be upgraded to persistence using browser caches
- Thematic design: A consistent UI design language
- Buried point design: page performance monitoring, request monitoring, operation statistics, error statistics, user area statistics and so on
- Highly available page Accessibility. Element compatibility, user availability, peripherals such as mouse and keyboard support, supported browser platforms, degradation schemes, flexible layout, terminal adaptation, weak network environment detection and degradation, and more
- Dynamic effect: skeleton screen scheme, loading animation scheme, general transition effect
- HTTPS and HTTP2.0 (3.0) support
- Multi-cdn support, Gzip support
- Multi-level cache scheme. HTTP cache, DNS cache, localStorage, indexDB, and more
- SPA / MPA / SSR
- Anti-crawler scheme
- Other web page performance optimization solutions
The above scheme is any website can consider the scheme, generally through these technical means, our web access experience can achieve good results.
Targeted program
Of course, only answering some general solutions may not give you much advantage, because you may only be listing some nouns, only targeted to deal with some questions, can reflect your thinking.
-
How to handle image files?
- File upload. Most of the current images are very large, do they involve breakpoint uploads? Subcontract upload?
- File download. Of course, the skeleton screen is a solution, and so is the multi-level image size, such as 500px. The images displayed in the list are all compressed non-original images. Only when you have enough permission can you access the original images through second view, download and other operations. This is where the user experience comes in.
- File preprocessing. Modern JavaScript engines are so fast that we can even use browsers to preprocess images, taking the pressure off the server side. Even some generic algorithms can be packaged as WASM on web pages using WebAssembly technology as a means of image preprocessing.
- Whether to use waterfall flow or random blocks also involves some simple graph-dependent algorithms
-
What if there is a network problem?
- This is where the caching scheme comes in. For example, indexDB, if used properly, allows us to store many of our public images on users’ computers. Or use localStorage to store the URL of the image and reduce the number of requests sent to the back end (because the image is usually stored on a specific server or CDN).
- PWA can also be used as an alternative, allowing web access to behave like a native app.
- Degrade. After the network speed slows down, the interface may time out. After the timeout, the interface can try again, and the user can be reminded of network problems.
These are just two questions that you can ask yourself if there are other questions that you can solve, for example
- How to anti-crawler?
- How to achieve the best image loading speed?
- How to realize image watermarking function?
- Can image compression and classification be realized in the front end?
4. Network design
At the network level, there are security and stability concerns.
- Network availability
- Network access speed
- Standby function, disaster recovery processing
- Data security, prevent network attacks
- HTTPS/http2.0 / HTTP 3.0 / websocket/long polling
In addition, any server deployment form, the collection becomes a topological network, each access request ultimately landed on which server, is the need for specific design processing. Network security, as an application deployed on the public network, must be paid special attention to.
- Server security, anti-intrusion
- https
- Client attack defense
- Anti-crawler scheme
5. Design the server
Server design also needs to consider security and stability. In addition, basic interface functions, permissions, algorithms and other solutions also need to have some concrete examples, that is, you give the solution can solve some problems.
- Basic data interface. Need anti-crawler, permission identification, tolerance of concurrency statistics, not just because a little bit more traffic will slow down other services. Therefore, it is possible to deploy the basic data interface as a microservice separately.
- Algorithm module. First of all, what is the purpose? First, image compression and decompression, such as filter algorithm needs to be efficient, reduce the hardware and software resource usage, the second, also can analyze user behavior, we can through the machine learning, analysis of the user’s behavior characteristics, such as a region of the people may be more like the outdoors, a city people prefer a dog rather than cat, and so on. How do you get features? You can set up a form for the user to upload, which is the most traditional solution. The more elegant solution is to collect the characteristics of the image itself and do machine learning in the background. All of which makes sense for algorithms to be the core competency of your site, given the rise of a cosmic company.
- Permission module. Permission restrictions can be divided into user level restrictions, also can be divided into operation type restrictions, also can be divided into regional restrictions and so on, permission module may be as the bottom of the cash flow of the website technology, is also worth paying attention to.
- Message communication. How to synchronize messages between microservices? How can a service failure not affect the user experience or reduce the risk of errors? And so on can be considered. And, as users grow and social networks become more complex, can your messaging handle the volume of data? We don’t have to design ahead of time, but at the very least, you have to design a system that can handle some of the peaks, even peaks plus 20% of the traffic.
- Log processing. Logs are also an important module to quickly locate problems or analyze user behaviors. But the log can not play more, key points of the log on the line. In addition, log files must be periodically cleared or backed up to avoid occupying too many system resources. Logging is also available as a separate service.
- Disaster recovery processing. At least have one master and one standby to make your system more reliable. Big companies might even have three or four backup servers. How to choose the backup time nodes of these servers? Too frequent affects machine performance, and the interval is too long. What if the service fails in the middle? These are real problems. Like a database, it is more likely that after a service access is complete, it sends a broadcast to another backup server, so that the backup server also stores it. Speaking of broadcasting, it comes back to message communication.
- Security. Network intrusion prevention, server account management, data security, and so on also need to be considered. Firewall policy, XSS attack defense, data security verification, file format verification and so on. For instance the user implanted a Trojan file to upload, burst your database, do not cry.
These are some of the points I’ve taken into consideration, but there are many others you can add.
As you can see, the point of architectural design is to constantly ask yourself, is the algorithm fast enough? What is the memory footprint? Magnitude of concurrent requests? Will the thread pool burst the stack? What if the service is down? What causes the service to hang? Is server performance being maximized? And so on and so forth, because in system design, it’s about continuous dialogue to reach your potential. Come on, you’ve done a lot of thinking so far.
6. Database design
Because the author mainly writes the front end, the data aspect knowledge is really deficient (needs to strengthen). Here are just a few possible points to ponder
- Depots storage
- The query efficiency
- Storage efficiency
- Data synchronization
- Data security
- Disaster preparedness to deal with
These questions need some professional answers. This is just a piece of cake.
summary
The above mentioned points to consider on the client, network, server, database, and so on. Of course, these are only summary summary, each sub-problem should be combined with specific problems for specific analysis. For example, you always talk about designing your architecture based on traffic. How do you count traffic? How do you collect real server running status (CPU | mem | disk)? When these are spread out, they can be turned into books again. So I want you to look at any subproblem as a thinker, and learn to ask yourself, is that the only way to do it in this module? Or is it the best it can be?
In addition, I want to emphasize the customer experience.
Although the core of system design is software and hardware design and network design. However, user experience is also important. If you are skilled in software architecture design, when someone asks you how to design a function, please pay attention to everything from the customer, the first thing is to understand the needs of the customer, and any software and hardware design is to improve user experience and truly meet the needs of the customer. You can get a lot more out of building your software architecture on this basis.
7. Engineering
The reader may assume that when the hardware and software design is complete, the system design is over. Software design and coding may be over (temporarily), but the problem is far from over. At least I can ask you those questions
- Once the architecture is designed, how do you ensure the quality of your software? (multistage test | code review | commissioning)
- How do you respond quickly to the constant demands of your users after the first release of your product is delivered? (Agile development | Team vitality | Architecture scalability)
- Software is deployed to the server, how do you do the later operation and maintenance? (Server capacity expansion | Backup | Log collection | Performance monitoring)
- The number of customers has reached a certain point, how do you ensure the security of customer data? (Social engineering attack | virus | malicious attack | defense drill | security awareness)
There are many other problems that can be identified, and readers can ask themselves, can they handle them? For example, how to bring a small team, a large team? How to carry out technical breakthrough?
8. After-sales service
This is the end of it. The product has been sold, and our service is good, but there are always customer feedback problems, how to solve them?
- Service unavailable? (server crash | attack | DNS contamination | algorithm crash | concurrency problem | log overload | carrier problem…)
- Slow website access? (Client network conditions | carrier network conditions | server processing efficiency | concurrent processing capability…)
- Bad user experience? (Permission design problem | UI design problem | service unreachable | font | image blur | recommendation algorithm…)
summary
Engineering and after-sales service, is not only quality assurance and supervision, but also the digging point of new demand. The daily work is also the most contact (in most cases only a small part of the development time, most of the time we are in the maintenance and optimization of our application), life everywhere learned, please pay attention to the reader.
Put an end to
Thank you for taking a few minutes to read my paragraph, which is not only the summary of my systematic interview, but also the thinking direction of my future work. After writing this paragraph, I have to feel that I still need more experience and training to become a qualified engineer.
It’s a long time. We work together.
Original address (public account article, one more mind map)
Original is not easy, thanks for reading