Make a mobile webApp running through the front and back end of 'Douban rental'

One, foreword

After learning React recently, I always want to do a project, but I have no good idea. Because they also want to rent, I thought of the idea of renting App, referring to douban renting small program, started such a simple front and back end project 😄. 👉 online demo click here 👉 project source click here, you can download in the local run, if it helps you can click star ha 😁

// You can use NPM or YARNYarn Install runs your database// Must!!
yarn server // Run the server. The connected database is configured in config.js under the server directory
yarn start // Run the project
Copy the code

Second, the front end

Project technology stackreact+react-router+react-reduxwithcreate-react-appScaffolding generation. UI adoptionAnt Design Mobile👉The official address.
cssThe scheme is CSS-in-JS, adoptedstyle-jsx, 👉Making the addressSee an article on the Nuggets at 👉Click here to.
Because it’s mobile, it can’t be avoidedadapterProblem, adoptvm/vhAdaptation, specific can also refer to the nuggets 👉This article.
In combination with2, 3,Two o ‘clock, due to add configuration items, but I don’t want in the projectrun ejectPop it out, so I used itreact-app-rewiredRewrite the configuration so you don’t have to pop up commands. 👉Making the address
Permission routing. The idea is to traverse the route configuration table, and use the permission route for those requiring permission, and use the original route for those not requiring permission. See the Router section of the project for details.
ICONS usediconfont SVGTo deal with

Specific usage ViewThe official address.

Problems encountered:

(not resolved) Changes to CSS in SCSS under dev development environment do not compile updates in real time
(Unresolved) Passed in IOSfocusThe event does not evoke the keyboard, android does. In order to have a better user experience, I’m inLog in, searchMake the input box automatic when the page opensfocusEvoke keyboard, classicsIOSReal machine practice, can only triggerfocusEvent, but does not invoke the keyboard,The androidNormal. After consulting the information isIOSDo the limit, (IOSAlso, audio and video can’t play automatically. After the user clicks the input box, the keyboard can be aroused. Next time reopen can automatically arouse the keyboard, very pit of a bit 😒!Currently no solution🙄
(solution)Click details from the home page list and return to the home pageRerequest loadAnd,Loss of scroll positionThe user experience is very poor. In order to learnreduxI useredux(also availablereactThe newcontext api), so it is also used in routingreact-redux-router, but has not maintained, insteadconnect-react-routerMaking here. The idea is to get the listing for the first time and then save itreduxThe next time you open it, fromreduxIn the acquisition.
(solution)Thermal loadingUnable to save afterreduxState in. Solution: InstoreAdd the following code,Look at this in detail

if (module.hot) {
// Reload reducers
    module.hot.accept('./reducers', () => {
        store.replaceReducer(connectRouter(history)(rootReducer));
    });
}
Copy the code

(Solve??)With thereact-loadbleLoad the search page in the project and there will be a searchinputtheplaceholderIf the display is incomplete, there will be a problem when opening it for the first time, but no problem when opening it for the second time, as shown below. indevCan’t be reproduced in the environment,The production environmentThere will be problems next time. The component ofAnt Design MobilethesearchBar.

In the figure above, we can see that it is the width problem110pxAnd when it’s wrong80px.Temporary solution: Remove the lazy route and directly load 😏

Project optimization:

Route lazy loading, scheme:react-loadableTo addloadingprompt
Lazy loading of images, scheme:lazyload
- Package into a component 👉 concrete code
- Here it needs to be noted that the pictures of douban loaded on your website are403So we need to use the following website to load the imageClick here to, method of usehttps://images.weserv.nl/?url=+ the original addressRefer to the link in the previous code for details
ajaxAdd as necessaryloadingPrompt, addCSS3Animation, make interaction more friendly.

Third, the back end

usingkoa2+koa-router+mongodb+jsonWebToken. The main thing is to pay attentionAsynchronous and exception handlingThe problem.
I used it for the databaseMongooseTo operate.MongooseThis is done in the node.js asynchronous environmentmongodbObject model tool for easy manipulation. For more details, please refer to the official documentation: 👉Click here to

3.1 Climb douban group data

Use ofhttpThe library isaxios.
Scheduled task library,node-schedule. Making: 👉Click here to
The crawler librarycheerioIt is very simple to use.

const cheerio = require('cheerio')
const $ = cheerio.load('<h2 class="title">Hello world</h2>')
Copy the code

So here we can get the elements of the page with $(selector), just like jquery. Official document: 👉 Click here

The entire crawl process:

When initializing, determine if the data length is greater than the maximum stored data (maximum stored database is set in this project5000Piece of data), ifMore than“Is deleted. Otherwise, the deletion is skipped.
Start a scheduled task every day0.00 amBegan to crawl= >Crawl the list page= >Database entry= >iffailure, will not crawl the TID
, or continue to climb the details page.
The crawler used some regular expressions to extract information, such as rent, contact information, house type, location and so on. Details: 👉 Click here. Some of the regular expressions in the 👉 article are referenced.

Note: Douban will limit the number of times the Ip can be accessed during a period of time, so we need to make some adjustments.

Each page of the list page and each piece of data on the detail page are guaranteed to have a different time interval for crawling. Timer + random number time

// sleep
function sleep(time = 0) {
 return new Promise(resolve= > {
   setTimeout(resolve, time);
 });
}
 // Update the database function
 async updateTopic(tid, resolve, reject) {
   / / sleep
   await sleep(Math.ceil(Math.random() * 50 * 1000) + 5000);
   // Start updating
   await this.fetchDetail(tid).then(houseInfo= >{... }); }Copy the code

To change the request headeruser-agent. There is one in the programuser-agentList 👉Look at the codeEach request carries a random one.

3.2 Storing the Database

Here I am inserting multiple pieces of data at once, using the following API

db.Houses.insertMany([your array data])
Copy the code

3.3 Write Interface (Routing)

Note that the header of some routes (interfaces that can be accessed only after users log in to them) requires token transmission. Therefore, the middleware of the route is added to verify that the access is allowed only after passing the verification. See the detailed code here. The key code is 👇

const jwt = require('jsonwebtoken');
const token = ctx.header['x-token'];
if(token){parse token to get user information into the next middleware}else{return error required to pass token}Copy the code

4. Mogodb related

4.1 Modifying database structures

When I started designing the database, I set the prices field to be an array. Prices =>price ()

Batch update a field

db.getCollection('houses').find().forEach(function(item){
    db.getCollection('houses').update({_id:item._id},{$set: {prices: ' '+item.prices}})
 })
Copy the code

Changing the field name

// If "prices" is changed to "price"
db.getCollection('houses').update({},{$rename: {'prices':'price'}}, false.true)
Copy the code

4.2 Attach some apis.

Database replication. For example, copy the Douban-house database to douban-test

// db.copyDatabase(<from_dbname>, <to_dbname>, <from_hostname>)
 db.copyDatabase('douban-house'.'douban-test')
Copy the code

Find data in the database whose array length is large/less than N

// Greater than EXISTS =1 less than exists=0
db.getCollection('houses').find({'imgs.n': {'$exists':1}})
Copy the code

Find data in a database where the field is not null

// $ne=> not equal
db.getCollection('houses').find({'contact': {$ne:null}})
Copy the code

Find multiple fields of data in a database

db.getCollection('houses').find({'tid': {$in: ['Here's an array'.'for example id1'.'2']}})
Copy the code

NumberInt is stored as a Double with a decimal point, for example, 10. When stored in the database, it is 10.0, and can be stored as NumberInt or NumberLong

db.houses.insert({"tid": NumberInt(Awesome!)})
Copy the code

4.3 Problems encountered

When a crawler crawls a post, it crawls to the same post, and we don’t need that duplication. The problem here is that when inserting duplicate values, the error does not continue to insert the rest of the data, which is a pitfall. Here’s how:

To set up the firstmongodbThe unique index value, in the setting of the time also encountered a lot of pits, checked a lot of information, summed up the relevantapi

const housesSchema = new mongoose.Schema({
	tid: String.// The only index I set here is the ID number of each post. Omit}) housesSchema. Index ({tid: 1 }, { unique: true });
Copy the code

When a duplicate TID is inserted, the database returns an error and does not insert that tid. Something special needs to be saidholeIs inserted into the API either wayinsertorinsertMany, theirapiThe following

db.collection.insert(
  <document or array of documents>,
  {
    writeConcern: <document>,
    ordered: <boolean>
  }
)
Copy the code

Note here that the 'ordered' parameter, which is an optional one, is officially explained belowCopy the code

Optional. A boolean specifying whether the mongod instance should perform an ordered or unordered insert. Defaults to true.

Specify whether the Mongod instance should perform an ordered insert. The default is true. ** When ordering an insert, if an error occurs, the program stops the current insert and does not perform the remaining data. Only when unordered inserts, with ' 'ordered: false' 'set, will the remaining inserts continue after an error occurs. The official instructions are as follows:  > Excluding Write Concern errors, ordered operations stop after an error, while unordered operations continue to process any remaining write operations in the queue. Official document links: 👉 [click here] (http://docs.mongodb.com/manual/reference/method/db.collection.insertMany)Copy the code

5. Deployment related (cross-domain processing)

The development phaseAvailable in the projectpackage.jsonaddproxyField, assumed herehttp://localhost:3003It’s our backend server,http://localhost:3000The react server was used during the react developmenthttp://localhost:3000/api/house/125048127It’s going to be represented byhttp://localhost:3003/api/house/125048127, there is no cross-domain problem

  "proxy": {
    "/api": {
      "target": "http://localhost:3003"}}Copy the code

Online environment nginx configuration agent 👉 see here

location  /api/ {
   proxy_pass   http://localhost:3003;
}
Copy the code

Git related

Sometimes when you commit the wrong code and want to roll back the version, you need to roll back the remote Git repository code and recommit. 👉 can be used here for more

git reflog // Check the submission list if I need to go back to the second submission record, the one under the red line
git reset --soft 3a2a12d // The soft parameter is soft, and the hard parameter is hard, and the hard parameter is soft, and the hard parameter is soft, and the hard parameter is hard. Remember!!
git push -f // Force push to remote branch
Copy the code

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Make a mobile webApp running through the front and back end of ‘Douban rental’

One, foreword

Second, the front end

Problems encountered:

Project optimization:

Third, the back end

3.1 Climb douban group data

3.2 Storing the Database

3.3 Write Interface (Routing)

4. Mogodb related

4.1 Modifying database structures

4.2 Attach some apis.

4.3 Problems encountered

5. Deployment related (cross-domain processing)

Git related

Make a mobile webApp running through the front and back end of ‘Douban rental’

One, foreword

Second, the front end

Problems encountered:

Project optimization:

Third, the back end

3.1 Climb douban group data

3.2 Storing the Database

3.3 Write Interface (Routing)

4. Mogodb related

4.1 Modifying database structures

4.2 Attach some apis.

4.3 Problems encountered

5. Deployment related (cross-domain processing)

Git related

Related Posts

CSS selector summary

Handy snippets of JavaScript code and tips

Recoil – Facebook official React status manager