An authorised person of the translation and the original address: https://medium.com/google-design/google-photos-45b714dfbed1
I had the good fortune to join the Google Photos team as an engineer a few years ago and worked on the first release in 2015. Countless designers, product managers, academics, and engineers (across platforms, front and back) are involved, to name just a few of the main responsibilities. I was responsible for the Web UI part, or more specifically, the grid layout of the photos.
We set our sights on the perfect layout: full-screen adaptive, scalable, easy to interact with (for example, users can jump to a specific location), and high performance and fast page loading while displaying a large number of images.
At the time, there was no album product on the market that could do all of these things. As far as I know, there hasn’t been anything comparable to Google Photos yet. Especially in terms of page layout and image proportions, most products still cut images into squares to ensure a beautiful layout.
Below I’ll share how we met these challenges, along with some of the technical details from Google Photos for the Web.
Why is this task so difficult?
There are two big ‘size’ related challenges.
The first ‘size’ challenge comes from the sheer volume of images (some users have uploaded more than 250,000 images) and the amount of metadata stored on the server. Even the amount of information a single image conveys (e.g. image URL, width, height, timestamp…) Not many, but because there are so many images, the page takes longer to load.
The second ‘size’ problem is the image itself. Even on modern HIGH-DEFINITION screens, a small photo is at least 50KB, and 1,000 of them are 50MB. Not only would the server be slow to transfer data, but even worse, the browser would crash when rendering so much content at once. In the early days of Google+ Photos, the browser tabs would freeze when loading 1,000 to 2,000 images, and crash when loading 10,000 images.
Here’S a four-part review of how we solved these two problems:
-
“Stand alone” picture – Quickly locate to a specified location in the picture gallery.
-
Adaptive layout – Spread the image as wide as possible according to the browser width and keep the original scale of the image (no square cropping).
-
Smooth scrolling at 60fps — keep the page interaction smooth even with the huge amount of data.
-
Timely feedback – Minimizes load time.
1. “Stand alone” pictures
I’m sure you’ve seen a lot of data presentation schemes. The most traditional pagination, for example, shows a fixed number of results on each page. Click “Next page” to get new data, and then go back and forth to see all the results. Now the more popular method is infinite scrolling, loading a quantitative amount of data at a time, and automatically pulling new data when the user scrolls the page near the end of the current data and inserts the page. If the process is smooth enough, you can scroll down the page forever — so-called infinite scrolling.
But there’s a problem with both paging and infinite scrolling: after all the data has been loaded, if the user wants to find the first photo — a nightmare.
For most pages, users can also navigate through scroll bars. For pagination, however, the scroll bar is at best located at the bottom of the current page, not at the bottom of the entire gallery; Infinite scrolling, the position of the scroll bar is always changing, and you can’t touch the bottom with the scroll bar until all the data is transferred to the client.
Separate image grids provide another way of thinking, in which scrollbars behave normally
In order for the user to use the scroll bar to navigate to the specified location, we need to reserve the page space. It would be nice if all of a user’s photos could be uploaded at once. The problem is that there’s too much data to do all at once. Looks like we need to try something else.
This is also a problem that other photo libraries face, and a common solution is to cut all images square in order to advance layout. This method only needs to know the total number of images: divide the viewport width by the determined square space size to get the number of columns, and then get the total number of images to get the number of rows.
const columns = Math.floor(viewportWidth / (thumbnailSize + thumbnailMargin));
const rows = Math.ceil(photoCount / columns);
const height = rows * (thumbnailSize + thumbnailMargin);
Copy the codeThree lines of code will do it, less than 12 lines of code will do the layout.
To reduce the first transfer of metadata, we came up with the idea of splitting the user’s photos into separate modules, passing only the module name and the number of photos under each module when first loaded. For example, partition modules with a “month” dimension — this step can be done on the server side (that is, calculated in advance). If the amount of data reaches millions, it can even be counted in “decade”. The data used for the first load would look something like this:
{
"2014 _06" : 514,
"2014 _05" : 203,
"2014 _04" : 1678,
"2014 _03" : 973,
"2014 _02" : 26,
// ...
"1999 _11" : 212
}
Copy the codeIf a user (such as a photographer) can produce a large number of images in the same time frame, this approach is flawed — the reason for breaking up the data into modules is to handle metadata easily, but for heavy users, the amount of data per month is still huge. The great base services team came up with a solution that allowed users to create custom categories (locations, timestamps…). .
With this information, we can place each module. When the user quickly scrolls the page, the client retrieves the image metadata, calculates the complete layout, and updates the page.
On the browser side, after getting the module’s metadata, we will rearrange the photos by date. The dynamic grouping we discussed (i.e. by location, person, date…) Would also be a great feature.
Now it is very simple to estimate the size of the module. After the number of photos and the proportion of the estimated single photo, calculate:
// Ideally, we should first calculate the average ratio of the current modules
// But let's assume the photo ratio is 3:2,
// Then make some adjustments based on it
const unwrappedWidth = (3 / 2) * photoCount * targetHeight * (7 / 10);
const rows = Math.ceil(unwrappedWidth / viewportWidth);
const height = rows * targetHeight;
Copy the codeAs you might guess, such estimates are inaccurate or even off by a wide margin.
I made things complicated at first (more on that in layout), but it turns out that it’s not necessarily necessary to get the exact numbers right at first (even thousands of pixels off in the case of large numbers of photos). The reason we do the estimation is to ensure that the scrollbar is in position, and it turns out that even this rough scrollbar position still works.
The trick here is that when the module is actually loaded, the browser will know the difference between the required and estimated space height by simply moving the rest of the page down by that distance.
If the module to be loaded is above the viewport, the scrollbar position needs to be updated after the module is loaded. All update operations can be completed in a second with a single animation frame, the impact on the user is not much, if the speed is fast enough users will not be aware of it.
2. Adaptive layout
As far as I know, the mainstream image adaptive layout on the market uses a clever and simple method: each row has a different height but fills the viewports, and images in the same row are scaled by their aspect ratio to ensure the height of images in the same row. Users are also less likely to notice the difference in height between rows.
Instead of making all images the same height, keep the original image in proportion and then fix the spacing between images. It’s not too hard to do. Find the highest row, scale each image by aspect ratio, update the current mesh width, and scale down each image in the row if it’s going to exceed the viewport width.
For example, when there are 14 images:
This method is very cost-effective, Google+ used to use this method, Google Search is a modification of this method, but the same idea. Flickr optimized their solution (they further compared whether it was better to have one less image or one more when they were about to exceed the viewport width) and made it open source. The simplified version is as follows:
let row = [];
let currentWidth = 0;
photos.forEach(photo => {
row.push(photo);
currentWidth += Math.round((maxHeight / photo.height) * photo.width);
if (currentWidth >= viewportWidth) {
rows.push(row);
row = [];
currentWidth = 0;
}
});
row.length && rows.push(row);
Copy the codeAt first, I was (unnecessarily) worried that the estimate was far from the final one and made things more complicated. But along the way, I stumbled on a solution.
My idea is: picture grid layout and text folding problem is similar. Using the fully documented Knuth & Plass line folding algorithm, I decided to apply it to image layout.
Unlike text folding, in image layout we think in terms of modules, where each line within a module affects the layout of the lines that follow.
The basic units of the K&P algorithm are box, glue and Penalty. Box is every non-separable block, which is also the object we need to locate. In the layout of the article, Box is a word or a single character. Glue are the Spaces between boxes, Spaces in the case of text, that can be stretched or compressed; To prevent the Box from being split twice, the concept of Penalty was introduced, which is usually a hyphen or a newline character.
Do you notice that the width of glues between boxes is variable?
Image folding is simpler than text truncation. In the case of text, people can accept a variety of truncation schemes — adding Spaces between words; Or increase word spacing; You can also use a hyphen. But in the picture scene, if the picture gap width is different, the user will notice; There is no such thing as a “picture hyphen.”
See here to learn more about the text folding algorithm, which will not be expanded in this article. Going back to images, we will use the algorithm just mentioned to do our image folding.
To apply this to image layout, we wanted to get rid of Glue, simplify the use of Penalty, and think of images as boxes. Having said that, it’s probably more accurate to say that we’re getting rid of Box and keeping Glue, imagining that it’s the images that vary in size, not their spacing. Or we can just assume that our Box is the same size.
Instead of changing the image spacing, we chose to adjust the height of the rows to adjust the layout. Most of the time, folding requires extra space. When you fold a line ahead of time, in order to fill the width, you increase the vertical space, because the original line needs to be higher; On the other hand, when a line is deferred, the height of the line is reduced. Calculate all the possibilities to find the most suitable size scheme.
Now there are only three things to consider: the ideal row height, the maximum compression factor (how low a row can be compressed), and the maximum stretching factor (or how high it can be stretched).
The algorithm works by examining each image one at a time for possible newline points — for example, when enlarging a group of images, they should be within a specified height (maxShrink ≤ image height ≤ maxStretch). Whenever you find a position that can be used as a line break point, note it down, and continue looking at that position until you have examined all images and all line breaks.
The 14 images below, for example, fit three or four images in a row. If the first line has three images, the second line break might be at the sixth or seventh image; If you put four cards in the first row, the newline in the second row will be in the seventh or eighth position. See, the newline of the previous line will determine the layout of the image that follows, but no matter where you cut it, it will always be grid layout.
The final step is to calculate the “badness value” of each line, or how unsatisfactory the current line break scheme is. For rows of the same height as our default, the bad value is 0; The more the row height is compressed/stretched, the greater this value is, in other words, the less desirable the layout of the row. Finally, the score for each row is converted into a value (called demerits) by some calculation. A number of articles have written about the formula, usually summing up bad values and then squaring or cubing them, plus some constants. In Google Photos we use a power of the sum to the maximum scaling value (the less ideal the row height, the larger the demerits will be).
The final result is a “graph”, in which each node represents a picture, which is the line breaking point, and each edge represents a row (a node may be connected to multiple edges, which means that there are multiple line breaking possibilities at the end of an image). We will calculate the value of each edge, namely the previous demerits.
For example, here are 14 images, we want each row to be 180px high, and now the viewport is 1120px wide. You can see that there are 19 line feeds (19 edges) that result in 12 different layout effects (12 paths). The blue line is the least bad way (I can’t say best). Follow these edges, and you’ll see that the bottom combination contains all layout possibilities, with no duplicated rows and no duplicated layout results.
Finding the optimal (or as optimal as possible) solution to the layout is as simple as finding the shortest path in the diagram.
Fortunately, we get a directed acyclic graph (DAG, where there are no duplicated nodes) so that the shortest path calculation can be done in linear time (which means “fast” to computers). But we can build the graph and find the shortest path at the same time.
To get the total length of the path, you just add the values of each edge together. Every time a new edge appears on the same node, check all its paths to see if a shorter total length value appears, and if so, write it down.
In the case of the 14 images above, the process of checking is as follows — the first line shows the image currently indexed (the first and last image in the row), the image below shows the newline points found and which edges are connected to them, and the shortest path on the current node is highlighted in pink. This is a variant of the figure above — each edge between the boxes is associated with a unique row layout.
Looking backwards from the first figure, if you set a newline point at index 2, demerits is 114. If the line break point is set at index 3, the demerits becomes 9483. Now we need to start from these two indexes and find the next newline point. The next step of index 2 is at 5 or 6, and the calculation shows that the path is shorter at 6 (114+1442=1556). The next step for index 3 could also be 6, but because the cost of line breaking at 3 was too high initially, demerits at 6 ended up being surprisingly high (9483 +1007=10490). So the current optimal path is to truncate at index 2 and then at index 6. At the end of the animation you will see that the path to index 11 chosen initially is not optimal, the one at node 8 is.
Repeat until the last image (index 13), at which point the shortest path (the blue path in the image above) is the best layout.
Below is a traditional layout algorithm on the left and a line folding optimization algorithm on the right. The ideal row height is 180px. After careful observation, two interesting conclusions can be drawn: traditional algorithms always compress the row height; The optimization algorithm increases the line height boldly. The end result is indeed that the optimization algorithm is closer to the ideal height.
In our tests, FlexLayout (we gave the image folding algorithm a name) actually produced a better grid layout. It produces a more uniform mesh (the height of each row is about the same) and the average row height will be closer to the preset height. Because FlexLayout allows for different permutations, extreme cases like panoramas can be solved. If the panorama is compressed to a very low level, the bad value of the edge in FlexLayout will be very high and the edge will definitely not show up in the final result. When a traditional algorithm encounters a panoramic (super-wide) photo, it treats it as an image in the first row and compresses it to be extremely short in order to fit it in the first row.
This means that some rows have different heights than the default, but not by much.
There are many variables that affect the final result: the number of images is one of the biggest influences; Viewport width and compression/stretch ratio are also important.
This is what FlexLayout would look like if it implemented a 25-image layout on narrow, medium, and widescreen screens. Newline points on a narrow screen don’t have much choice, but they produce a lot of lines. As the screen widens, the likelihood of newlines on the same row increases, and accordingly the number of lines decreases, as does the likelihood of layout.
As the number of images increases, the number of layout schemes increases exponentially. In a medium wide viewport, the number of paths for different images is as follows:
5 photos = 2 paths
10 photos = 5 paths
50 photos = 24136 paths
75 photos = 433144 paths
100 photos = 553389172 paths
Copy the codeWith 1000 images, the computer doesn’t have time to count the number of layout options, but miraculously it can immediately find the best path, even if it doesn’t have time to verify that the path is really the best.
However, the optimal layout can be calculated according to the formula, and the mean of the possibility of each line break point can be calculated, and then the cubic can be used to calculate the total possibility of the number of trips. For most viewport widths, there may be two or three line breaks per line, and more than five images can fit on one line. There are usually 2.5^(number of images /5) possible layouts.
A combination of 1000 pictures might have 100… 000 (79 0 species); 1,260 images have 10^100 possibilities.
Whereas traditional algorithms can output only one layout solution at a time, FlexLayout computs trillions of solutions simultaneously and selects the best one.
You must be wondering if the client/server side can handle this amount of computing, and the answer is “yes”. Calculating the best layout for 100 photos took 2 milliseconds; 1,000 photos in 10 milliseconds; 10,000 photos is 50 milliseconds… We also tested 100 million photos in 1.5 seconds. The traditional algorithm takes 2 ms, 3 ms, 30 ms and 400 ms respectively in corresponding scenarios, which is faster but less experienced than FlexLayout.
At first we just wanted to choose the best layout, but later we could fine-tune the mesh spacing so that the user would always see the best layout.
Everyone raved about FlexLayout, which is available for Android and iOS, and is now updated on all three platforms, including the Web.
Last but not least, each section is counted twice: the first is a single photo of the segment in the section, and the dimension is the photo; The second calculation is segment in section, dimension is segment. Because there may be too few segments or pictures, so that no line is filled, the second calculation is required. In this case, the layout algorithm will suggest merging the content with less than one line to achieve the best visual effect.
3. Scroll at 60fps
We’ve done a lot of work to optimize the layout so far, but if the browser can’t handle that much data, it’s all for nothing. Fortunately, the browser allows developers to optimize page rendering.
In addition to the first page load, users often experience “slowness” when navigating the page, especially scrolling. The browser is designed to draw at 60 frames per second (60 FPS), which is the speed at which the user feels comfortable navigating the page, and vice versa.
What does 60fps mean? Render time per frame should not exceed 16ms (1/60). But beyond rendering the content of the page, the browser has plenty of tasks — handling events, parsing styles, calculating layouts, converting all element units to pixels, and finally drawing — to leave at least 10 milliseconds.
In those precious 10 milliseconds, make sure you’re doing it efficiently, but also make sure you’re not wasting time.
Keep the DOM size the same
Too many elements can affect page performance for two reasons. First, the browser takes up too much memory (1000 50KB images require 50MB of ram, 10000 images take up 0.5GB, enough to crash Chrome). Another point is that having more elements means that the browser has more styling, layout, and composition to do.
Although users already have thousands of images stored in Google Photos, they only see one screen at a time, and in most cases only a few dozen.
We don’t think it’s necessary to load all the images on the page at once, but rather to listen to the user’s actions on the page and display the images in their position as they scroll.
Take out images that were previously visible but are now out of the viewport due to scrolling.
Even though the user has viewed hundreds or thousands of photos on the page, there are no more than 50 images to render at a time due to viewport limitations. In this way, the user’s interaction can always get timely response, and the browser is not prone to crash.
Fortunately, the images are grouped according to the segment and section dimensions in advance. Now there is no need to operate a single image, you can mount/suspend the whole module at once.
Variable minimization
There are plenty of great articles on rendering performance at Google Developers, as well as tutorials on how to use Chrome’s built-in performance detection tools. Here’S a quick overview of some of the techniques used in Google Photos, and please visit Google Developers for more details. Let’s start with the page rendering lifecycle:
Whenever a page changes (usually triggered by JS, but also by styles or animations), the browser first identifies which styles are responsible for the change, recalculates the element layout (size and position), and then redraws all the affected elements (such as text, images… Convert to pixels). To improve the efficiency of updating page content, browsers often divide elements into different layers, drawing them in layers, and the final step is layer composition.
Most of the time, the browser is smart enough that you probably won’t remember the render pipeline. But if the content of the page changes too often (such as constantly adding/subtracting images), be careful.
To minimize page variation, we position all child elements relative to their parent elements. Sections are absolutely located in the entire grid layout. Segments are absolutely located in relation to the section in which they are located. By analogy, an image is absolutely positioned in the segment to which it belongs.
After positioning all the elements, when we need to change the size of a section (the actual height and the estimated height are often different, such an update), all the elements below its physical location need to change the top value. This layout avoids a lot of unnecessary DOM updates.
The CSS contain attribute defines how much an element is contained, so that the browser knows how much it affects the rest of the context. So we add this attribute to both sections and segments:
/* The internal and external contents of the element do not interact */
contain: layout;
Copy the codeThere are also performance issues that are easier to deal with, such as multiple scroll events within a single frame and continuous scrolling when the browser window is zoomed. If the layout is constantly changing, the browser doesn’t have to recalculate the style and layout at the beginning of the change.
Fortunately, this default behavior can be through the window. The requestAnimationFrame (the callback) ban, this method is used in the next frame before implement the callback function. In the case of scrolling and scaling events, it allows us to perform callbacks first rather than update the layout directly; Window scaling is a little more complicated: perform the update half a second after the user determines the final window size.
The second common problem is layout jitter. When the browser needs to calculate the layout, it caches the layout first so that it can quickly find the width, height, and layout information later. However, once attributes that affect the layout are changed (such as width, height, top, or left… The previous layout cache is immediately invalidated; When the layout properties are read again, the browser forces the layout to be recalculated (multiple times within the same frame).
This can be problematic in scenarios with a large number of elements in a circular layout, such as hundreds of images. Reading a layout property changes the layout (moving an image or section to the correct position), and then reading a layout property triggers a new round of layout calculations.
A simple solution to avoid this problem is to read all the values at once and update them at once (i.e., separate the reads and writes and do batch processing). But our approach is to avoid reading values, record the size and location of each photo, and absolutely locate them. When scrolling or window scaling occurs, we perform all calculations based on the recorded photo information. This update method does not cause jitter. Here is the performance of the page after scrolling to update one frame (you can see that there are no duplicated links in the render pipeline) :
Avoid code running continuously
Thanks to Web Workers and support for native asynchronous methods such as Fetch, a TAB page has only one thread, meaning that the code in the same TAB page runs in one thread — both rendering and JS. This means that if code (such as a long-running scrolling event method) blocks the rendering of the page, the user will have a bad health check.
The most time-consuming part of our solution is creating layouts and elements. These two operations must be completed in a certain amount of time so that the user is not affected.
For example, 1000 image layouts take 10 ms and 10000 images take 50 ms, which takes up 60 ms of update time. But because we split images into sections and segments, it only takes 2-3 milliseconds to update a few hundred images at a time.
The most “expensive” layout event is the window scaling — each section has to be resized. We went back to the original algorithm — even if some sections were already loaded, we didn’t do anything about it. We only used FlexLayout for sections that were visible. Wait until other sections are scrolled into viewport range before recalculating.
The same logic is used to create elements — we do layout calculations just before the image is about to be seen.
The results of
After all this work, we ended up with a pretty good layout — 60fps in most cases, although there were occasional frame drops.
Dropping frames usually occurs during major layout scenes (such as inserting a brand new section) or when the browser wants to reclaim particularly old elements.
The feeling of the moment
I’m sure most front-end engineers put a lot of effort into the UI to show off their skills, such as fireworks and so on.
One of my favorites, the Watch out machine, was the brainchild of a YouTube colleague. The progress bar (the red bar at the top of the page) didn’t actually load the page (there was no actual progress information at the time), but they animated the “loading” experience until the red line reached the far right of the page. I’m not sure if YouTube now matches the loading animation to the actual loading progress of the page, but the whole idea is this.
The accuracy of the loading schedule is secondary to the fact that the user actually feels that the page is moving forward.
In this section I’m going to share some tips for making Google Photos feel smoother to use (and smoother than it really is) — most of them have to do with image loading.
The first thing, and probably the most effective, is that the content the user is most likely to see is loaded first.
After loading images in the viewport range, an additional screen of images will be loaded to ensure that the next time the user scrolls the page, they will immediately see the new image.
But with HDPI screens, where we need to load larger thumbnails, it’s harder to respond to all requests when scrolling quickly.
So we optimized the loading scheme — loading the next four or five screens of placeholders first, which tend to be very small, so they load immediately. When the images are about to be moved to the viewport, load the original image.
This means that if a user scrolls slowly through images at normal speed, he won’t see the photos loading beyond the viewport. But there are also scenarios where you can quickly scroll to find an image, and the user will see a thumbnail of the image and feel the general information.
There is always unnecessary work to be done in order to get content on the page, but at the same time to provide a smooth user experience, it is a complex tradeoff game.
We considered the following factors. First, check the scrolling direction of the page. Preload what the user is about to see. It also recognizes whether to load the original hd image based on the user’s scrolling speed. If the user is only skimming through the image, the original image is not necessary to load. Even when the page scrolls fast enough, even low-resolution placeholders don’t need to be loaded.
Whether you load an original image or a low-resolution placeholder map, there will be a scene for scaling the image. Today’s screens are almost always high-definition, and it is common to load an image twice the size of the space, then reduce it in half and place it in the right place (so that a pixel can actually carry twice as much information). For low-resolution placeholders, we can request very small resources with high compression rates (say 75%) and then zoom in on them.
In this fast asleep, leopard, for example, on the left side of the picture is fully loaded in the grid layout well we will see later (it has been reduced to like half of the actual image size), the right is a low-resolution placeholder figure (also be magnified to the placeholder size), when a user rapidly across will see such a placeholder.
Also note the file size of the image, the compressed HD thumbnail is 71.2KB, and the lower resolution placeholder is 889B after the same compression algorithm, which is only 1/80 of the original HD image! Conversion, a HIGH-DEFINITION original map of the flow top four pages of the placeholder map.
In exchange for a better user experience with less traffic, placeholders allow users to experience rich web content and provide visual references while browsing.
The last thing to consider is how browsers render low-resolution placeholders. By default, the browser does pixel smoothing when a small image is enlarged (image center), but it doesn’t look very good. It works much better if you use blur (far right). But filters have a huge impact on page performance, and if you filter hundreds of images at once, page performance is incredibly poor. So we went the other way and let the browser pixelate these images (like the far left), but I’m not sure Google Photos still uses this solution, it’s a bit of a makeover.
If you want the user to never see a low-resolution image (except for a scene that is really unavoidable like fast scrolling), especially when you are about to enter the viewport and the original HD image is about to replace the time intersection of the placeholder image, we used animation to make the transition (to avoid replacing the image directly). This is done by superimposing the placeholder onto the original image, and transitioning the placeholder from opaque to fully transparent when the original image needs to be displayed — one of the common transitions, as illustrated in Medium. Google Photos may now have removed this transition logic, but the process from empty grid to content may still use this effect.
This visual experience gives the user the impression that the image is loading, and the animation lasts for 100 milliseconds — enough time to load the original image. Here’s a slow animation for you to see:
Here’s another place to use this technique: expand thumbnails to a full-screen preview. When the user clicks on the thumbnail, we immediately start loading the original image. While waiting for the original image, the thumbnail is enlarged and positioned in the middle of the screen. When the original image is loaded, the original image is displayed by changing its transparency. Unlike thumbnail loading, this time only one image is used, so a blur filter is used (the pixelated experience is definitely not as good as a blur).
Whether it’s scrolling through images, or switching between thumbnail mode and full-screen preview mode, we always want the user to feel that the end result is not yet ready, but that the browser is working on the task. Contrary to this idea of interaction, when the user clicks on the thumbnail, there is no feedback or even a blank screen until the original image is fully loaded.
The same idea applies to empty sections. Our grid layout only loads sections when it needs to display them (there are some preloaded images as well). If the user drags the scroll bar directly, the section is not loaded yet. Although the space is reserved, the user is not prepared for what images and layout he or she will see when he or she navigates to this location.
To make scrolling more natural, we set the height of the reserved sections as the target row height, and filled them with colors to indicate placeholders. When loading started, sections looked like long gray rectangles (far left below). Recently, sections have been modified to look like rows and columns on the far right below, more like images. The section in the middle of the image below has been loaded but the image has not been rendered.
This image loading process is like tracking animal tracks, so try to discern these states the next time you use Google Photos.
Section placeholders are implemented using CSS instead of images, so there is no distortion or clipping even if the width and height are arbitrarily changed:
-
/* Before the section is loaded, the width ratio is 4:3 */
-
background -color : #eee;
-
background -image :
-
linear -gradient (90deg , #fff 0, transparent 0, transparent 294px, #fff 294px, #fff),
-
linear -gradient (0deg , #fff 0, transparent 0, transparent 220px, #fff 220px, #fff);
-
background -size : 298px 224px;
-
background -position : 0 0, 0 - 4px ;
Copy the codeThere are a few other tips, mostly related to optimizing the order of requests. For example, instead of requesting 100 thumbnails at once, we request 10 thumbnails in 10 batches. So if the user suddenly starts scrolling too fast, they won’t waste the next 90 pages. Similarly, images within the viewport are always requested first, images outside the viewport are requested slightly, and so on.
We even reuse thumbnails of similar size — for example, when the user scales the window, the grid layout doesn’t change substantially, just the number of rows. Instead of re-downloading the thumbnail of another size, we scale the existing image and request the image again only when the window size is completely changed.
conclusion
Google Photos takes a lot of user experience into account, and the grid layout is just the tip of the iceberg
At first glance it looks like a simple or even static layout, but the grid is always changing in real time — loading, prefetching, animating, creating, removing… To give the user the best experience it can.
The team always prioritizes ensuring and improving product performance. The Google Photos team has been able to use a combination of scrolling frame rates, module loading rates… Google Photos is always moving forward.
Here’s a screenshot of scrolling through the Google Photos page. As the user slowly navigates the page, they see clear thumbnails; When the scroll speed is increased, the pixelated placeholder is displayed. When the scroll speed is returned to slow scroll, the high-resolution image is displayed again. When you fly across the page, you see a gray placeholder. The scrolling speed different load effect (https://www.youtube.com/watch?v=AEpwAzLISXU) :
Thanks to Vincent Mo, my leader at Google Photos, who has been very supportive and who took all of the Photos used in this article (Vincent’s Photos were also used in the product testing phase). Thanks to Jeremy Selier, head of the Google Photos Web side, who is now leading the team that continues to maintain and improve the Google Photos Web side experience.