background

This article is from my previous post on Weibo:

This essay is intended to introduce you to the basics of computers and the Internet, and to give you an idea of how they are related.

To make it easier to understand, I’ve divided the whole process into six questions.

First question: What happens between entering the URL and the browser receiving it?

From touch screens to cpus

The first is “enter URL.” Most people’s first reaction will be the keyboard, but to keep up with The Times, here’s how touchscreen devices interact.

Touch screen a sensor, currently based mostly on capacitors, used to cover the display directly, but recently there have been three technologies embedded In the display. The first is the iPhone 5’s In-Cell, which reduces the thickness by 0.5 mm. The second is on-cell technology used by Samsung, and the third is OGS full fit technology favored by Domestic manufacturers. Read this article for details.

When a finger touches the sensor, some of the electrons are transferred to the hand, causing a voltage change in the area. Based on this change, the touch-screen controller chip calculates the location of the touch and sends the signal through the bus interface to the CPU pin.

Take the Nexus 5, which uses the Synaptics S3350B touchscreen controller with the I²C bus interface. Here are some examples of Synaptics touchscreen and processor connections:

The left is the processor, the right is the touch screen controller, and the SDA and SCL connection in the middle is the I²C bus interface.

Processing within the CPU

The CPU in mobile devices is not a single chip, but integrated with chips such as gpus, known as soCs (systems on a chip).

As mentioned above, the connection between the touch screen and the CPU is communicated through electrical signals, that is, the change of voltage level, just like the connection inside most computers, as shown in the sequence diagram below:

Under the control of the clock, the current passes through the MOSFET transistor, which contains n-type semiconductor and P-type semiconductor, through the voltage can control the circuit open and close, and then these MOSFET constitute the CMOS, and then by CMOS logic circuit gates such as “and” “or” “not”. Finally, addition, displacement and other calculations can be realized on the logic circuit gate, as shown in the figure below (from Computer Architecture) :

In addition to computing, the CPU also needs storage units to load and store data. This storage unit is usually implemented by flip-flops, called registers.

How to Build an 8-bit Computer based on transistors, diodes, capacitors, and other components. It supports simple assembly instructions and output of results, although modern CPU implementations are much more complex than this. But the basic principle is the same.

In addition, I have just started learning CPU chip implementation, so I will not confuse the children, interested readers please read the book recommended later in this section.

From CPU to operating system kernel

The touch controller sends an electrical signal to the corresponding pin of the CPU, which triggers the INTERRUPT mechanism of the CPU. In Linux, for example, each peripheral device has an identifier called the interrupt Request (IRQ) number. You can use the /proc/interrupts file to view interrupt request numbers for all devices on your system. Here are some of the results for Nexus 7 (2013) :

shell@flo:/ $ cat /proc/interrupts
            CPU0
  17:          0       GIC  dg_timer
 294:    1973609   msmgpio  elan-ktf3k
 314:        679   msmgpio  KEY_POWER
Copy the code

Since the Nexus 7 uses ELAN’s touch controller, the Elan-KTF3K in the result is the touch interrupt request information, where 294 is the interrupt number and 1973609 is the number of times it was triggered (two interrupts when a finger is clicked, but hundreds of interrupts when a finger is swiped).

In order to simplify the priority issues are not considered here, for the ARMv7 architecture as well as tuning the processor, for example, when an interrupt occurs, the CPU will stop the currently running program, save the current execution state (such as PC value), the state of IRQ), and then jump to the corresponding interrupt handler implementation, the program general by a third party kernel driver, For example, the aforementioned driver source code for the Nexus 7 touch screens is in here.

The driver will read the location data from the I²C bus, and then record the touch screen press coordinates via the kernel’s input_report_ABS method. Finally, the input module in the kernel will write this information into the /dev/inpu/event0 device file. For example, here is the information generated by a touch event:

130 | shell @ Flo: / $getevent - lt/dev/input/event0 [414624.658986] EV_ABS ABS_MT_TRACKING_ID 0000835 c [414624.659017] EV_ABS ABS_MT_TOUCH_MAJOR 0000000B [414624.659047] EV_ABS ABS_MT_PRESSURE 0000001D [414624.659047] EV_ABS ABS_MT_POSITION_X 000003F0 [414624.659078] EV_ABS ABS_MT_POSITION_Y 00000588 [414624.659078] EV_SYN SYN_REPORT 00000000 [414624.699239] EV_ABS ABS_MT_TRACKING_ID FFFFFFFF [414624.699270] EV_SYN SYN_REPORT 00000000Copy the code

From the OPERATING system GUI to the browser

As mentioned earlier, the Linux kernel already abstracts the hardware, and other programs simply listen for changes to the /dev/input/event0 file to know what touches the user is making, but it would be too much trouble for every program to do that. Therefore, GUI frameworks are included in graphical operating systems to facilitate application development, such as the famous X under Linux.

Instead of using X, Android implements a GUI framework of its own, including an EventHub service that listens for files in the /dev/input/ directory via epoll. This information is then passed to Android’s WindowManagerService, which looks for the app based on the location information and calls its listener functions (such as onTouch).

So that’s the answer to the first question, but due to time constraints, many details have been omitted here, and the following books are recommended for further study.

Extend the learning

  • Computer Architecture
  • Computer Architecture: Quantitative Research Methods
  • Computer Composition and Design: Hardware/Software Interfaces
  • “The code”
  • Introduction to CPU Homemade
  • Operating System Concepts
  • Armv7-ar Architecture Reference Manual
  • Linux Kernel Design and Implementation
  • Proficient in Linux Device Driver Development

Second question: How does the browser send data to the network card?

From the browser to the browser kernel

As mentioned earlier, the OPERATING system GUI passes the input event to the browser. During this process, the browser may do some pre-processing. For example, Chrome will use historical statistics to predict the web site of the entered character, such as “BA”. Based on our previous history, we found that 90% of the time we would visit www.baidu.com, so we would immediately set up TCP links and even render them before typing back to the car. There are a lot of other strategies, High Performance Networking in Chrome is recommended for interested readers.

Then enter “Enter” after the URL, at this time the browser will check the URL, first determine the protocol, if it is HTTP according to the Web processing, in addition to the URL security check, and then directly call the corresponding method in the browser kernel, For example, the loadUrl method in WebView.

In the browser kernel, the cache is first looked at, then HTTP information such as UA is set, and then the methods of network requests on different platforms are called.

It should be noted that browser and browser kernel are different concepts. Browser refers to Chrome and Firefox, while the browser kernel is Blink and Gecko. The browser kernel is only responsible for rendering, while GUI and network connection and other cross-platform work are implemented by the browser

Sending of HTTP requests

Because the underlying implementation of the network is related to the kernel, so this part needs to be processed for different platforms. From the perspective of the application layer, it mainly does two things: query IP through DNS and send data through Socket. The following two aspects will be introduced respectively.

The DNS query

Applications can implement DNS queries directly by calling the liBC-provided getAddrInfo () method.

Dig +trace fex.baidu.com DNS query is implemented based on UDP, here is a concrete example to understand its search process, the following is the result of using the dig +trace fex.baidu.com command (omitted some) :

; <<>> DiG 9.8.3-P1 <<>> + fex.baidu.com; global options: +cmd . 11157 IN NS g.root-servers.net. . 11157 IN NS i.root-servers.net. . 11157 IN NS j.root-servers.net. . 11157 IN NS  a.root-servers.net. . 11157 IN NS l.root-servers.net. ;; Received 228 bytes from 8.8.8.8#53(8.8.8.8) in 220 ms com.172800 in NS a.gtld-servers.net.com.172800 in NS c.gtld-servers.net. com. 172800 IN NS m.gtld-servers.net. com. 172800 IN NS h.gtld-servers.net. com. 172800 IN NS e.gtld-servers.net. ;; Received 503 bytes from 192.36.148.17#53(192.36.148.17) in 185 ms baidu.172800 in NS dns.baidu.baidu.com. 172800 IN NS ns2.baidu.com. baidu.com. 172800 IN NS ns3.baidu.com. baidu.com. 172800 IN NS ns4.baidu.com. baidu.com. 172800 IN NS ns7.baidu.com. ;; Received 201 bytes from 192.48.79.30#53(192.48.79.30) in 1237 ms fex.baidu.com. 7200 in CNAME fexteam.duapp.com. fexteam.duapp.com. 300 IN CNAME duapp.n.shifen.com. n.shifen.com. 86400 IN NS ns1.n.shifen.com. n.shifen.com. 86400 IN NS ns4.n.shifen.com. n.shifen.com. 86400 IN NS ns2.n.shifen.com. n.shifen.com. 86400 IN NS ns5.n.shifen.com. n.shifen.com. 86400 IN NS ns3.n.shifen.com. ;; Received 258 bytes from 61.135.165.235#53(61.135.165.235) in 2 msCopy the code

It can be seen that this is a search process of narrowing the scope step by step. First, the DNS server set on the local machine (8.8.8.8) queries the domain server responsible for.com zone to the DNS root node, and then queries the server responsible for Baidu.com through one of the servers responsible for.com. Finally, one of the domain name servers of Baidu.com queries the address of the fex.baidu.com domain name.

You may find that when you query some domain names, you will see that some strange server returns results first.

For example, 127.0.0.1 is loopback and has nothing to do with the nic device. Chrome, for example, looks up 10 domains you are likely to visit when the browser launches. In addition, the Hosts file and the cache Time TO live (TTL) are affected.

Send data over the Socket

If you have an IP address, you can use the Socket API to send data. In this case, you can choose TCP or UDP.

TCP does not block HTTP and does not block HTTP. TCP does not block HTTP and does not block HTTP. Suppose the client sends three TCP segments numbered 1, 2, and 3. If the packet numbered 1 is lost during transmission, the client has to wait for the packet numbered 2 and 3 to arrive, because THE SEQUENCE is guaranteed by TCP. This problem is exacerbated by HTTP Pipelining, which allows multiple HTTP requests to be sent over a single TCP connection, such as sending two images that have already been received while waiting for the first image to be sent.

To address TCP’s performance issues, the Chrome team last year introduced QUIC, a reliable udP-based transport that reduces round trip time significantly compared to TCP. There are also features such as Forward Error Correction. QUIC is now used in Google Plus, Gmail, Google Search, Blogspot, Youtube, and almost all of Google’s products. It can be found on the Chrome ://net-internals/#spdy page.

While no one uses QUIC outside of Google yet, I think it’s promising because optimizing TCP requires kernel upgrades (such as Fast Open).

Browser on the same domain are limited number of connections, most is 6, I used to think that will change the number of connections will be big performance, but in fact is not the case, the Chrome team has done the experiment, found from 6 to 10 after performance fell instead, there are many factors that can cause this phenomenon, such as the overhead of a connection is established, the problem such as congestion control, Protocols like SPDY and HTTP 2.0 use only one TCP connection to transfer data, but perform better and prioritize requests.

In addition, because HTTP requests are in plain text format, HTTP text can be directly analyzed in TCP data segments, if found…

Socket in the kernel implementation

The browser’s cross-platform library calls the Socket API to send data. How is the Socket API implemented?

Take Linux as an example, its implementation is here in socket.c, which I am not familiar with at present. I recommend readers to look at the Linux Kernel Map, which notes the function of critical path for learning from the protocol stack to the implementation of nic driver.

Concrete examples of underlying network protocols

If you continue to introduce IP and MAC protocols, many readers may be confused, so this section will use Wireshark to explain through specific examples. The following is the network data captured when I request baidu home page:

The actual binary data is at the bottom, and the parsed field values are in the middle. You can see that the Hypertext Transfer Protocol (HTTP) is at the bottom, with 54 bytes (0x36) before HTTP. This is the overhead brought by the underlying network Protocol. We examine these protocols next.

On top of HTTP is the Transmission Control Protocol (TCP), as shown in the following figure:

From the binary data at the bottom, you can see that TCP is superimposed on the HTTP text and is 20 bytes long, TCP defines the Source port, Destination port, Sequence Number, and window length. The following is a complete description of TCP data:

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Acknowledgment Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Data | |U|A|E|R|S|F| | | Offset| Reserved |R|C|O|S|Y|I| Window | | | |G|K|L|T|N|N| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum | Urgent Pointer | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Options | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Copy the code

The specific functions of each field are not introduced here, but interested readers can read RFC 793 and analyze captured packets to understand.

Note that there is no IP address information in TCP because this is defined in the previous IP protocol, as shown in the following figure:

The Version number is 4. The Source IP address is 192.168.1.106. The Destination IP address is 119.75.217.56. Therefore, the most important function of IP protocol is to determine the IP address.

Because the IP protocol can see the destination IP address, if certain IP addresses are found, some routers will…

The IP address is not bound to a certain device. For example, the IP address of your laptop is 192.168.1.1 at home, but changed to 172.22.22.22 at work. Therefore, you need to use a fixed address for underlying communication. This is the MEDIA Access Control (MAC) address. The MAC address of each nic is fixed and unique before delivery.

So on top of that is the MAC protocol, which has 14 bytes, as follows:

When a computer joins a network, it uses THE ARP protocol to tell other network devices its IP address and MAC address. In this way, other devices can find the corresponding device based on the IP address.

Frame indicates the number of the captured packet, not the network protocol

In this way, we have solved the second question, but there are many, many details that have not been introduced, I recommend you to further study through the following books.

Extend the learning

  • Computer Networking: The Top-down Approach and Internet Features
  • Computer Network
  • The Definitive Guide to Web Performance

Third question: How is data sent from the local network card to the server?

From kernel to Network Interface Card

After calling the Socket API, the kernel encapsulates the data from the underlying protocol stack, and then starts the DMA controller, which reads the data from memory and writes it to the nic.

The Nexus 5, for example, uses a Broadcom BCM4339 chip to communicate with an SD-card-like SDIO interface, but the details of this chip are not publicly available, so we won’t discuss them here.

Connect the Wi-Fi route

Wi-fi network card need via wi-fi routing and external communication, the principle is based on the radio, through changing electric current to generate the radio, the process is also called “modulation”, and in turn the radio can cause electromagnetic field change, resulting in a current change, can use this principle to the information in the radio reading out call “demodulation”, The number of changes per unit time is called frequency. Currently, the frequency used in Wi-Fi is divided into 2.4ghz and 5ghz.

In the same Wi-Fi route, because the same frequency is used, there will be conflicts when they are used at the same time. To solve this problem, Wi-Fi adopts a method called CSMA/CA, which simply means to confirm whether the channel has been used before transmission and send data only when there is no channel.

2G/3G/LTE, which is also based on radio, has a similar problem, but it doesn’t use a wi-fi-like proprietary solution, but uses frequency division (FDMA), time division (TDMA), and code division (CDMA) for reuse. The details are not covered here.

Taking Xiaomi Routing as an example, it uses BCM 4709, which is composed of ARM Cortex-A9 processor and Flow hardware acceleration. Using hardware chips can avoid operating system interruption, context switch and other operations, thus improving performance.

The operating system in the router can be developed based on OpenWrt or DD-WRT, but I don’t know the details, so I won’t go into that.

The IP address of the Intranet device is 192.168.1.x, and the Internet cannot directly send data to this IP address. Therefore, the route modifies the IP address and port when the network data passes through the route. This operation is called NAT mapping.

Finally, home routing is usually connected to the carrier network via twisted-pair cables.

Routes within the carrier network

After the twisted-pair data is sent to the carrier’s network, it is forwarded through many intermediate routes. You can use the traceroute command or the online visualization tool to view the IP addresses and locations of these routes.

When data is transmitted to these routers, the router takes out the prefix of the destination address in the packet and searches for the corresponding output link through the internal forwarding table. How is this forwarding table obtained? This is the most important route selection algorithm in a router, there are a lot of options, I don’t know much about it, it looks like wikipedia lists it all.

Transmission between backbone networks

For long-term data transmission, usually using optical fiber as medium, optical fiber is implemented based on the total reflection of light, the use of optical fiber require specialized launcher by electroluminescent (such as LED) to electrical signals into light, compared with the previous introduction of radio and twisted-pair cable, optical fiber signal anti-jamming better, and the energy consumption is much smaller.

Since the data is transmitted based on light, the data transmission speed also depends on the speed of light. The light speed in vacuum is close to 300,000 km/s, and the refractive index in cladding is 1.52, so the actual light speed is about 200,000 km/s. The distance from Beijing Capital Airport to Guangzhou Baiyun Airport is 1,967 kilometers, which takes about 10 milliseconds to arrive. This means that if you are in Beijing and the server is in Guangzhou, it will take at least 20 milliseconds for you to send the data back to the server. The actual situation is expected to be 2-3 times as long as it takes to process the routing of each node. For example, I tested an IP in Guangzhou and found that the average delay is 60 milliseconds.

This delay is beyond current technology (unless a way is found to exceed the speed of light) and can only be achieved by CDN to shorten the transmission distance or minimize serial requests back and forth (such as the 3 handshakes required by TCP to establish a connection).

IDC network

Data through the optical fiber will eventually come to the server in the IDC room, into the IDC Intranet, then you can first through the optical splitter to mirror a flow out to facilitate security inspection and other analysis, can also be used to carry out…

The cost of bandwidth here is very high and is settled according to the peak value, in units of Gbps per month (note that this refers to bit, not Byte). In Beijing, the price is more than 100,000 RMB, and the average website uses 1G to 10G.

The data from the fiber goes to the Cluster switch and is then forwarded to the switch at the top of the Rack. Finally, the data is sent to the servers in the Rack through the ports on the switch, as shown in the following figure (from Open Compute) :

The image above shows the front on the left and the side on the right. You can see the space left for the switch at the top.

In the past, the internal implementation of these switches was closed, and related manufacturers (such as Cisco, Juniper, etc.) would use specific processors and operating systems, which made it difficult for external flexible control and even manual configuration sometimes. However, with the popularity of OpenFlow technology in recent years, There is also Open Switch Hardware, such as Intel’s networking platform, which is recommended for interested readers to watch videos that are much clearer than the text description.

Note that switches mentioned in general network books only have layer-2 (MAC protocol) functions, but switches in IDC basically have Layer-3 (IP protocol) functions, so there is no need for special routes.

Finally, because the CPU processes electrical signals, the light in the fiber needs to be converted into electrical signals by the photoelectric effect using the relevant equipment, and then into the server network card.

Server CPU

The nic copies the data to memory (DMA) and then interrupts it to notify the CPU. Currently, the CPU on the server side is basically Intel Xeon, but there are some new architectures in recent years. For example, in the storage domain, Baidu uses the ARM architecture to increase storage density because it has much lower power consumption than Xeon. In the area of high performance, Google is currently experimenting with servers based on the POWER architecture CPU. The latest POWER8 processor can execute 96 threads in parallel, so it should be useful for high-concurrency applications.

Extend the learning

  • The Datacenter as a Computer
  • Open Computer
  • Software Defined Networks
  • Big Talk Wireless Communication

Fourth question: What does the server do when it receives the data?

To avoid duplication, I’m going to skip the operating system and jump right into the back-end services process, but because there are so many technology choices, I’ll just pick a few common parts.

Load balancing

Before a request is sent to a real application server, it may also pass through a load-balancing machine that distributes the request to multiple servers and has anti-attack capabilities.

There are many specific implementations of load balancing, including F5 directly based on hardware, LVS on the operating system transport layer (TCP), and reverse proxy (also known as layer 7 proxy) implemented at the application layer (HTTP). Next, LVS and reverse proxy will be introduced.

There are many load balancing strategies. If the performance of multiple servers is balanced, the simplest method is to Round Robin one by one. Other strategies are not introduced one by one, you can refer to the algorithm in LVS.

LVS

The function of AN LVS is to have only one IP, but in fact this IP corresponds to multiple machines, so it is also called a Virtual IP.

NAT mentioned above is also a working mode in LVS, in addition to DR and TUNNEL, the specific details will not be expanded here, their disadvantage is that they cannot cross network segments, so Baidu developed the BVS system.

The reverse proxy

The reverse proxy works on HTTP. The specific implementation can be based on HAProxy or Nginx. Because the reverse proxy understands the HTTP protocol, it can do a lot of things, such as:

  • A lot of unified processing, such as anti-attack policy, anti-fetching, SSL, GZIP, automatic performance optimization, etc
  • The traffic diversion policies of the application layer can be implemented here, for example, the requests to/XX paths are sent to server A, the requests to/YY paths are sent to server B, or small traffic tests are conducted according to cookies
  • Cache and display a friendly 404 page when the backend service fails
  • Monitor whether back-end services are abnormal
  • …

Nginx code is very well written, and there is a lot to learn from it that readers interested in high performance server-side development should look at.

Processing in Web Server

After the previous load balancing, the request will be sent to the Web Server on the corresponding Server, such as Apache, Tomcat, and Node.js.

Take Apache for example, after receiving the request, it will be handed to an independent process to process, we can write Apache extension to process, but this is too troublesome to develop, so it is generally called PHP scripting language to process, for example, in CGI is to put HTTP parameters into environment variables, Then start the PHP process to execute, or use FastCGI to prestart the process.

(I’ll talk about node.js separately later when I have time)

Enter the backend language

Now that the Web Server calls a backend language process to handle HTTP requests (which is not entirely true, there are many other possibilities), it’s time for backend languages to handle HTTP requests. Most of the backend languages are currently based on virtual machines. PHP, Java, JavaScript, Python, etc., but the topic in this field is very large and difficult to explain clearly. Readers who are interested in PHP can read my previous introduction to HHVM, which mentioned a lot of virtual machine basics.

Web Framework (Framework)

If your PHP is just a simple “Personal Home Page”, there is no need to use a Web framework, but if the code becomes more and more difficult to manage, so most websites will be developed based ona Web framework. Therefore, when executing in a backend language, the code of the Web framework is first entered, and the implementation code of the application is then called by the framework.

There are so many Web frameworks to choose from that I won’t cover them here.

Read the data

This section is not expanded, from simple reading and writing files to the data middle layer, there are too many options.

Extend the learning

  • Nginx in Depth
  • Python Source Code Anatomy
  • In-depth Understanding of the Java Virtual Machine
  • Database System Implementation

Fifth question: What does the browser do with the data returned by the server?

We start with this section by looking at How Browsers process data when they receive it. One thing to note is that there is a good article on this topic, How Browsers Work, so I don’t want to start with many things again. So focus on the parts that were overlooked in that article.

From 01 to characters

When the HTML returned from the HTTP request is delivered to the browser, if there is a gzip, it will extract it first, and then the most important question is to know what its encoding is, such as the same “middle” word, In UTF-8 encoding, its content is actually “11100100 10111000 10101101”, namely “E4 B8 AD”, and in GBK it is “11010110 11010000”, namely “D6 D0”, how to know the file encoding? There are many ways to judge:

  • User Settings, page encoding can be specified in the browser
  • In the HTTP protocol
  • <meta>The value of the Charset property in
  • For JS and CSS
  • For the iframe

If none of these is specified, it is difficult for the browser to process as a group of “zeros” and “ones” as it sees, such as “Chinese” which is 6 bytes as “trick- as well as others as UTF-8.

But normal people can recognize at a glance “trickling water  枃” is wrong, because these three words too not common, so someone is thought of by judging method to detect the encoding of common word, a typical such as Mozilla UniversalCharsetDetection, but this dongdong misjudgment rate is also high, So it’s better to specify the code.

Subsequent operations on text are then based on Character, where a Chinese Character is a Character, regardless of whether it is two or three bytes long.

Loading of external chain resources

(To be added, here is the scheduling policy)

The execution of JavaScript

(I will introduce it separately later, and recommend you to read this post organized by R University last year, which has a lot of relevant information. In addition, I talked about performance optimization in JavaScript engine two years ago, although some content is not quite correct, you can also read it.)

From characters to pictures

The most complex 2d rendering is the text display, although it seems very simple to think about it, isn’t it to find out the corresponding glyph of a certain text? In Chinese and English this is ok because each character corresponds to a glyph (glyph) and you can just find the glyph in the font file and draw it, but not in Arabic because it has a conjoined form.

(To be covered separately later, this is very complicated.)

Cross-platform 2D drawing library

Each operating system provides its own rendering API, such as Quartz for Mac OS X, GDI for Windows, and Xlib for Linux, but they are not compatible with each other, so in order to facilitate cross-platform drawing, The Skia library is used in Chrome.

(I’ll cover this separately later, because there are so many levels of internal Skia calls that it may not be suitable for beginners to talk directly about the code)

GPU synthesis

(It will be introduced separately in the future. Although it simply depends on texture, OpenGL and GPU chip will also be introduced. The content is too long.)

Extend the learning

This section is the content I am most familiar with, but the results because it just want to spend more time to write, so wait until later to send out good, we can first take a look at the following sites:

  • Chromium
  • Mozilla Hacks
  • Surfin’ Safari

Question 6: How does the browser render the page?

As mentioned earlier, the browser has rendered the page as an image, so the next problem is how to display this image on the screen.

Framebuffer

On Linux, for example, the most straightforward way to control the screen in an application is to write a bitmap of the image to the /dev/fb0 file, which is actually a map of a memory region called the Framebuffer.

Note that under hardware acceleration, OpenGL does not go through the Framebuffer.

From memory to LCD

When the Framebuffer is ready, the CPU notifies the LCD controller via the AMBA internal bus. The controller then reads the data in the Framebuffer and performs format conversion, gamma correction and other operations. Finally through DSI, HDMI and other interfaces sent to LCD display.

Take OMAP5432 as an example. Here is a parallel data transfer supported by OMAP5432:

The LCD display

Finally, the display principle of LCD is briefly introduced.

First, for the human eye to see, light must come in, either through reflection or from a light source. For example, the E-Ink screen used on the Kindle doesn’t emit light itself, so you have to be in a light source to read.

At present, LED is usually used as the light source in LCD. After the LED is connected to the power supply, under the action of voltage, the combination of positive and negative electrons inside the LED will release photons, thus generating light. This physical phenomenon is called Electroluminescence, which was also introduced in the previous introduction of optical fiber.

Here’s what the iPod Touch 2 looks like when taken apart:

In the image above, you can see six leds, which are the light source for the entire screen, and these light sources will be output to the screen through the reflected reflections.

A light source also has to have Color, in LED is usually the use of Color filter (Color filter) to the LED light source into different colors.

In addition, it is also feasible to directly use three-color LED, which can avoid photon waste caused by light filtering and reduce power consumption. It is very suitable for small screens like smart watches. LuxVue, a company acquired by Apple, adopts this approach

Each physical pixel on an LCD screen is actually made up of red, green, and blue dots, and each color dot can be controlled individually. Here’s what it looks like under a microscope (Wikipedia) :

From the image above, you can see that every three color filters are white when they are all on, and black when they are all off. If you look closely, you can also see that some dots are not completely black, which is the anti-aliasing effect on the font.

Various colors can be generated by different combinations of the brightness of these 3 colors. If each color point can produce 256 brightness, 256 * 256 * 256 = 16777216 colors can be generated.

Not all monitors can reach 256 brightness. When selecting a monitor, one parameter is an 8-bit or 6-bit panel. The 8-bit panel can physically reach 256 brightness, while the 6-bit panel has only 64 brightness. It relies on refresh rate control technology to achieve 256.

How to control the brightness of these color dots? This will depend on LCD, LCD feature is when there is a current through the rotation happens, which will be part of the light blocking, so as long as through the rotation of the voltage controlled liquid crystal can control the brightness of the color point, the current mobile phone screen is typically used in TFT controller to control, the most famous is the IPS in TFT panel.

Most of this filtered light goes directly to the eye, and some of it has gone through the Diffuse or specular reflection of ambient light on other surfaces. To really figure out how much light reaches the eye is an integral matter for interested readers to explore physics-based rendering.

Once light enters the eye, it’s the realm of biology, so we’ll end there.

Extend the learning

  • Computer Graphics, 3rd Edition: Principles and Practices
  • Interactive Computer Graphics

Omitted in this article

For ease of writing, many low-level implementation details were omitted in the previous introduction, such as:

  • Memory related
    • Heap, where there are many allocation strategies, such as malloc implementation
    • Stacks, function calls, there have been many good articles and books written about them
    • Memory mapping, dynamic library loading, etc
    • Queues are almost everywhere, but these details have little to do with principle
  • All kinds of cache
    • CPU caches, operating system caches, HTTP caches, back-end caches, etc
  • All kinds of monitoring
    • Many logs are saved for subsequent analysis

FAQ

Judging from the feedback on Weibo, some questions are frequently asked, so I will answer them here. If you have any other questions, please ask them in the comments.

Q: What’s the use of learning so much? Not at all

A: The computer is man’s most powerful tool. Don’t you want to know how it works?

Q: How about mastering one rather than knowing a little of everything?

A: I agree with you very much. At the beginning, you need to be proficient in A certain field, and then you need to learn about the surrounding field. This can also give you A deeper understanding of the previous field.

Q: To cultivate a bunch of noodle bullies against themselves?

A: This article is actually very shallow. Each part can be expanded further.

Q: This problem is exhausting. I can’t finish talking for several days

A: Ha ha ha, god, you are exposed. The title is just A means to dig out A big bull like you.

Everybody’s discussion

Thank you very much for participating in the discussion. Here are some of the answers.

WOODHEAD CLUMSY: Requests are sent to local routing, access vendor routing, bypass analysis for illegal addresses, the connection is broken, and the browser innocently says the page doesn’t exist. Serious someone to check the water meter

Caoz: That was my interview question! There is also a question, the user response to our website card, what are the possibilities, and the platoon method.

Caoz: It’s good, but there are a few flaws, like ARP spoofing? The famous GFW blocking strategy, and the fact that a URL is not just one request, multiple requests queuing and addressing? In addition, CDN, intelligent DNS resolution mechanism and so on. // @zrj -: http://t.cn/8smHpMF from click to present — a full description of an HTTP request I wrote in my junior year. ah

Tang Fulin: Keep pace with The Times, it’s time to ask what happens during the whole process from opening the app to refreshing the content. If it feels slow, how to locate the problem and how to solve it

@ winter winter: Reply @Ivony: This question wins because of its high degree of differentiation, even coverage of knowledge points, and people who don’t know much can also answer a few sentences, while masters can freely play according to their own areas of expertise. From URL specification, HTTP protocol, DNS, CDN, streaming resolution to browser, CSS rule construction, Layout, Paint, onload/ domReady, JS implementation, JS API binding

@JS group :[ha ha] Xiaobian remember, it seems that the most beautiful sister @sherrie_Wong in the front-end industry asked xiaobian this question in the interview when she first started working in the industry. And then I told you everything I knew at the time, parsing from the browser, making requests, the model of the layer 7 network in practice, the TCP three-way handshake. Routing, switch,DNS, to the server. Whether you need to deal with file systems or databases, and distributed computing hadoop… Too much talking.

Rapunzel: This is a gangster question. I also want to ask what happens between the time you press the keyboard and the time a character appears on the screen. Hint: imagine you’re an electron. Oh, no, what’s an electron

@ winter winter: http://t.cn/zH20bR1 http://t.cn/zH20bR1 wrote two articles in the beginning, before the waste at the end of…

@ILS Rumor: Not to mention the generator of the power plant has been turned several times also killed! // @philonis Gao: Not to mention the complete elimination of how switches and routers work! // @Spider: From the point of view of the 7 layer agreement, it will be more comprehensive. This is the kind of question only full stack engineers can answer.

@Shrugged Atlas :DNS parses the URL out of IP/Port, the browser connects and issues a GET request to this address, the Web server (Nginx, Apache) receives the request, through CGI and other interface protocols to call dynamic language (PHP, etc.), dynamic language and then connect to the database to query the corresponding data and process, It then feeds back to the browser, which interprets the page, processes it with HTML, javascript, CSS, and displays it on the screen… It’s an 800-page book for each detail

1 Zhao Lingyan: This kind of interview questions in the system field of recruitment is actually simple and good. What is the process between typing a character key on the keyboard and displaying it in terminal?

ICT_ Zhu Yadong: I remember 6 years ago when I took Hu Weiwu’s chip design class, Old Hu said in the first class, AFTER this class, I hope you can make it clear. I turned over a page of PPT and found that all the flow operations were done inside the computer. Of course, I don’t remember at all.

Julyclyde: What happens to a TCP segment in an IP packet in an Ethernet frame when it passes through a router

@sicyfesi: Starting with browser, OS corresponding keyboard interrupts, event queue processing, to Internet routing, to server nic interrupts, to output buffering… You can talk about it for two hours.

Xscape: Start with the keyboard break? Back to the car before the pre-analysis is very late.. //@ Pure white burning: from keyboard to spring to gravity and then to quantum mechanics.

Bosn: Then from hardware to electronics… quantum… Schrodinger’s Cat… Parallel universes… and even universal philosophy!!

ImPony: can penetrate into the electron flow level of PN junction

Gong Xiaodong -TX: The browser group sends HTTP packets through sock, filters them through proxy, receives processing headers, and returns unexpired cache. The HTTP SVR processes verification packets, converts them into CGI protocols, and sends them to the back end. The back end map URL, load code, and generates HTML for the SVR after interacting with logic. SVR filters cache to proxy, proxy to browser, pulls JS to complete HTML, and browser rendering.

@yuange1975: I think I know the whole process quite well, including server handling, Web server and browser handling and security issues, I guess few have studied the security of both. But it is also difficult to list out the bulk of the process clearly and completely during the interview. It is probably difficult to have the opportunity to sort out the articles.

Wang Lei: I have been asking this question for many years, or a variation: from entering URL to presenting, what are the caching links involved, and what is the updating mechanism of caching

1 Zhao Lingyan: This kind of interview questions in the system field of recruitment is actually simple and good. What is the process between typing a character key on the keyboard and displaying it in terminal?

@Wisdom fool: It can be said in different dimensions, mainly depending on the granularity, optical network from wifi decryption, to NAT, to inter-office switching, IP packet mapping in Ethernet packet and so on can write a book

/ @jo3: All Internet related knowledge can be reflected, such as DNS, browser cache, TCP connection, HTTP response, how web services work, browser response and rendering, etc., just listed the security threats in the book, very interesting!

The last

Careful readers should find hidden content in this article, please find…