The 2017 alipay bill really lived up to everyone’s expectations, once again flooded the screen.
Looking back at this year, there are three phenomenon-level screen-brushing activities:
Show “18”; Show netease Music playlist; Show your Alipay bill.
A netizen joked: “This is the beginning of 2018 large-scale” dating “show…
Show “18 years old”, look at the level of appearance;
Show netease music playlist, sun taste;
Show off your alipay bills and show off your wealth.
Aren’t the three important indicators of “blind date” complete? !
All joking aside, as a programmer, I have “occupational disease” again. Just like the “jump jump” small program produced by wechat, all kinds of plug-ins and brush points are coming in an endless stream, occupying the headlines of major technical media.
So I decided to explore the technology behind alipay bills. But this does not mean that I will create another Alipay bill, after all, the core part of this bill is the massive consumption data behind, which is not available to me.
Due to the limited level of personal experience, if there is any inadequacy of the conjecture made in this paper, please correct. If ant Financial students are willing to share the technical architecture behind the billing, it would be very welcome.
This paper is a personal guess of alipay billing technology realization, I will be simple and rough into two parts: front end, back end. From the outside to the inside, inferring the invisible back-end logic from the visible front-end display.
First, looking for a breakthrough
The first step of the whole exploration process is to find the entrance, which I think is a very critical breakthrough. I shared the bill via Studs, then went to Studs FOR PC and right clicked the share record to copy the entire URL.
For the sake of explanation, here is the entire URL:
https://render.alipay.com/p/s/i/?scheme=alipays://platformapi/startapp?appId=68687017&showOptionMenu=NO&allowsBounceVert ical=NO&transparentTitle=auto&bizScenario=Share&url=https://render.alipay.com/p/f/fd-jbg7if4k/index.html
Copy the codeThe function of the whole scheme parameter is to try to open the Alipay application on the mobile phone, which is easy to understand for students who do mobile development. Depending on the security policy of the browser, these are the results in Safari(iPhone) and Chrome(PC) :
Scheme comes with 6 parameters, which will be passed into APP to perform corresponding operations. I thought the scheme parameters shared by each user would be different, but after a comparison, I found that they were all the same parameters. After a study of the source code, found that the “bizScenario” parameter will be different, because I found a breakthrough from the nail Share, so the value of bizScenario is Share, but if it is opened from other channels, the value is not necessarily the same.
As for user information, it should be obtained in the page through JS interaction with Native. If we want to see the final bill page, we can simply copy the URL parameters from Scheme and open it in a browser.
At this point, we have opened a door to explore alipay bills.
Two, front-end page implementation
First of all, I need to understand that Alipay billing is a Single Page Application, in other words, it is a Single Page Application implemented by HTML + JS. What I have learned is static background image + cutscene + data layer. This not only solves the cross-platform problem, but also facilitates propagation.
It is worth mentioning that the entire application of CSS and JS files, including resource request URL, have done some encryption and confusion, to read, or there is a certain difficulty, especially JS code.
There are nine static background images, and I’m going to put three here, just to give you a sense of it.
A video
As for the source of the data and the layers that are created using JS, the following screenshot from the Chrome console shows the main DOM element of the page.
It is roughly divided into three parts: loading, bill body content, error prompt.
There are two main elements in the loading process, one is the loading animation, which is actually a calendar flip GIF animation; The other is the percentage of progress, which, without thinking about it, is a false progress indicator. Through ** “AlipayJSBridge” **, Native APP is entrusted to send the bill data request. In this process, the progress indicator increases at a certain rate, and it will stop at about 97% until the data is obtained, and then the bill page is officially entered.
The main content of the bill consists of three parts: The first one is swiper, which slides the screen to switch scenes. There are 9 sub-elements and 9 static pictures under it, corresponding to 9 scenes respectively. The second and third sections are the two video tabs that play the MP4 files of the next scene and the last scene respectively.
There’s nothing to explain about the error message, a line of text and a retry button.
Some readers may ask, how did you get such resources as pictures and videos? That’s what I’m going to talk about.
Let’s start with the definition of a data structure:
// Static resources.
window.resource = [
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/epRpbpBcCZIKasROmxcL.jpg"."video": {
"forward": ""."back": ""
},
"__key": 9
},
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/epRpbpBcCZIKasROmxcL.jpg"."video": {
"forward": "https://gw.alipayobjects.com/os/rmsportal/KwIPQNAHVfCMQSToOqxX.mp4"."back": "https://gw.alipayobjects.com/os/rmsportal/zRAHVrBufLRAlmOMwXgA.mp4"
},
"__key": 1
},
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/ALzmXZZYrnFDqYFFrGjY.jpg"."video": {
"forward": "https://gw.alipayobjects.com/os/rmsportal/QuZvhHxfIyRqgNxGQRqq.mp4"."back": "https://gw.alipayobjects.com/os/rmsportal/zRAHVrBufLRAlmOMwXgA.mp4"
},
"__key": 2
},
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/ZcmJnyzRQNuFspDzfoxX.jpg"."video": {
"forward": "https://gw.alipayobjects.com/os/rmsportal/CLryDglMNEQLfDxmYnUW.mp4"."back": "https://gw.alipayobjects.com/os/rmsportal/nmqWCcwURUxRdUqgdJje.mp4"
},
"__key": 3
},
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/qQLBJCGEDtXCoCiOtPzc.jpg"."video": {
"forward": "https://gw.alipayobjects.com/os/rmsportal/TUKbyXyonamHXwQifpDQ.mp4"."back": "https://gw.alipayobjects.com/os/rmsportal/rwcPCPShQgxvVYfobSeU.mp4"
},
"__key": 4
},
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/zQzcNkGFJJqHinrABCDa.jpg"."video": {
"forward": "https://gw.alipayobjects.com/os/rmsportal/OBocUmcqGaHuZJJkNQvV.mp4"."back": "https://gw.alipayobjects.com/os/rmsportal/HbyqRtKZMlKFQZcxwCJy.mp4"
},
"__key": 5
},
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/JzzNsINqFRVOCAMNJonO.jpg"."video": {
"forward": "https://gw.alipayobjects.com/os/rmsportal/kBYTpmZElHykGKYytIIG.mp4"."back": "https://gw.alipayobjects.com/os/rmsportal/zvFLfvTTgXdUUCFHuIlv.mp4"
},
"__key": 6
},
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/vOpPBFXzjbvlSAxZBcSB.jpg"."video": {
"forward": "https://gw.alipayobjects.com/os/rmsportal/dIPIDFjkwlJkxwjbtmRO.mp4"."back": "https://gw.alipayobjects.com/os/rmsportal/VIbUIjvACyUlBSVULXbB.mp4"
},
"__key": 7
},
{
"scene": "https://gw.alipayobjects.com/zos/rmsportal/HJUaMfRgdvNwrxJgibsJ.jpg"."video": {
"forward": "https://gw.alipayobjects.com/os/rmsportal/egQUtbPUbknxskiWkGgU.mp4"."back": "https://gw.alipayobjects.com/os/rmsportal/JlBuZvsTbtOGIiUDNTuW.mp4"
},
"__key": 8."poster": "https://gw.alipayobjects.com/zos/rmsportal/guBrTMglMdSRsWcRnaXB.jpg"}];Copy the codeThis code comes from the billing page, copied from the Chrome console.
Scene D. Forward animation, back animation; Poster seems to have little significance (from the perspective of implementation) and can be considered as a “spare” for the scene; As for the __key field that deserves the most consideration, I’ll give you my understanding of this field in the code below.
Pay special attention to the last group of scene objects, the video content in Forward seems not to appear in the whole bill, one of them accidentally let me see the name of the outsourcer of animation production, is this the “chicken leg” added by the program Monkey brother?
Note also the meanings of the values forward and back. The value of the forward field is easier to understand, that is, the current scene slides to the next scene, while the value of the back field is slightly more ambiguous. Let me directly use an example: in the __key=3 scene object, the back field records the scene of the __key=2 cutscene.
Because the JS source code is confused is really unreadable, can only be based on the effect of interaction to guess the code implementation. Here is the code for the scene switch:
// It is also possible to assign a value of 0 directly, guessing the function of __key in order to use this field
var index = window.resource[0].__key;
var len = window.resource.length;
function next() {
if(index == window.resource[len - 1].__key) return; //at the last page
index = (index + 1) % len;
var resource = window.resource[index];
//1. play video in resource.forward.
//2. video player listener binding.
//3. to display data with animations after forward video completed.
}
function previous() {
if(index == window.resource[0].__key) return; //at the first page
var resource = window.resource[index];
//1. play video in resource.back.
//2. video player listener binding.
//3. to display data with animations after back video completed.
//4. set value for index variable.
index = (index == window.resource[1].__key) ? window.resource[0].__key : index--;
}
Copy the codeMainly through the len module, the scene before and after switching, so as to achieve the effect of loop playback. The conjecture for __key field is also reflected in the code, it is worth mentioning that, in all THE JS source code search, did not find any place to use this field, DO not know whether it is confused by encryption.
This concludes the presentation of the front end.
Third, back-end data integration
This part will focus on the formation of alipay bill data. The conjecture is purely based on my understanding of The technical structure of Alipay, and does not represent the real operation situation.
The above is a reasonable architecture diagram in my opinion. The perspective of architecture is placed in the Data level.
1. Database: This layer represents cloud Database cluster. The databases in the whole cluster are likely to be heterogeneous, such as MySQL, Oracle, PostgreSQL, MongoDB, etc. In addition, the cluster mentioned here also covers the databases used by Taobao, Tmall, Alipay and other alibaba products. So this part carries a lot of data input and output work, is crucial.
2, DW, Data Warehouse, namely Data Warehouse. An important source of data will be the cloud database cluster, but some will come directly from files. There are many functions that can be realized in the data warehouse, among which ETL is the only way to BI. In cooperation with Reporting System, it can realize data visualization, log analysis, operation and maintenance monitoring and other functions.
3. MaxCompute, which is actually a large-scale distributed computing platform belonging to Aliyun, is represented by distributed computing frameworks such as Hadoop and Spark. Hadoop is good at offline computing, while Spark can complete fast real-time computing.
DRDS and REST APIs. DRDS is also a database middleware product produced by Ali Cloud. As mentioned above, cloud database cluster is heterogeneous, so one middleware must participate in data reading and writing. In the case of the REST APIs, it’s about providing a list of APIs for clients to manipulate data.
After explaining the entire architecture diagram, I walk through the entire data request process.
1. Data generation. It is mainly the user’s consumption record in 2017, which comes from tmall, Taobao, Alipay, Ant Credit and other platforms. Most of these data are stored in a structured database cluster.
2. Generate annual bill data. The user’s consumption data of 2017 is imported into the data warehouse, and the bill data of each user is calculated offline through the distributed computing platform, and the structured bill data is put into the database cluster. It is wise to use offline computing because the data is all petabytes and real-time computing is only for individual users, otherwise it will affect the user experience. This part of the work, simply put, is to write several MapReduce tasks, running on a distributed computing platform for 2 to 3 days should be about the same;
3. Data acquisition. At this point, the billing data is ready, and the client just needs to call the API to get it and make the billing page we see now.
Four, a little improvement
I made this architectural conjecture based on research into front-end displays, but it wasn’t the product of my first instinct. I always believe that “page static” technology can be used for marketing activities such as alipay bills and netease playlists. Of course, there are pros and cons to this architecture. Let’s take a look at an improved architecture diagram.
As you can see, the front end and back end are already in a highly decoupled state, with the back end only responsible for generating billing data and populating the user’s billing page, while the front end accesses the specified static HTML page. For this architecture, we will discuss the following three issues:
1. Naming of HTML files. That’s a lot of room to imagine, and the rules vary, such as simply Hash elements like user ID and generation time. Of course, there are also requirements for file directories, so I won’t go into the details here.
2, page static technology selection. Theoretically, the best choice is CDN technology, which has been relatively mature in the market and can be safely used. If you do not need CDN, squid can be considered to make a cache proxy cache service, which can be considered as a simplified version of CDN. If you only need content distribution without considering other more advanced functions, SQUID is a good solution.
3. Analysis of applicable scenarios. Static page attracted me most is that ease the pressure on a large amount of back-end data access, the pressure on to a CDN, but don’t have to worry about, because this is the CDN forte, make the cost is low, reach bottleneck, in addition, no additional data access network, not only will not expose apis, has certain security, front page can do seconds away, It brings excellent experience to users. Therefore, static pages are definitely not suitable for scenarios where pages change frequently or have strong interactions.
Originally the alipay billing page can be static, “license agreement” event happened afterwards let I dismissed the idea, the emergence of the episode means will generate good page before all failure and rectification, and will cause a big wave flow, can also cause waste of storage space, unless it is replacing the file before. Don’t think about it, Alipay bill page embedded dynamic authorization, it is not good to do the page static.
Five, the summary
In this paper, from the front end to the back end of two levels, the technical implementation of Alipay bill to do a very simple analysis, for some of the technical details can not be known, but also give part of their own implementation ideas. If you read this article and are interested in it, you can post your own thoughts on the topic.
Daily dry goods sharing, transmission of the Internet world valuable information, wechat public number: Technology