Prepare for spring recruitment/Autumn recruitment, open source community system Echo, based on the current mainstream Java Web technology stack, and provide detailed development documents and supporting tutorials

Source links: already hosted on Github and Gitee:

Github：github.com/Veal98/Echo
Gitee：gitee.com/veal98/Echo

Online experience: The project has been deployed to Tencent cloud server, you can directly online experience: http://1.15.127.74/. There are three built-in users with different identities:

	username	password	Special privileges
The administrator	admin	admin	Data statistics, delete posts
The moderator	master	master	Top of the post, and fine posts
The average user	user	user

Document address: Document generated by Docsify + Gitee Pages, domestic access speed is fast, online access address: veal98.gitee. IO /echo

📚 What can you learn from this project

Learn the mainstream Java Web development technology and framework (Spring, SpringBoot, Spring MVC, MyBatis, MySQL, Redis, Kafka, Elasticsearch, Spring Security, etc.)
Learn the whole process of a real Web project from development to deployment (there are plenty of legends and detailed tutorials to help you get started quickly)
Master the core technical points involved in this project, as well as frequent meeting questions and analysis

🏄 Online experience and document address

Online experience: The project has been deployed to Tencent cloud server, you can directly online experience: http://1.15.127.74/

There are three built-in users with different identities:

username password Special privileges

The administrator admin admin Data statistics, delete posts

The moderator master master Top of the post, and fine posts

The average user user user
This document is generated by Docsify + Gitee Pages. It can be accessed online at veal98.gitee

💻 Core technology stack

The backend:

Spring
Spring Boot 2.1.5 RELEASE
Spring MVC
The ORM: MyBatis
Database: MySQL 5.7
Distributed cache: Redis
Local cache: Caffeine
Message queues: Kafka 2.13-2.7.0
Search engine: Elasticsearch 6.4.3
Security: Spring Security
Mail task: Spring Mail
Distributed scheduled tasks: Spring Quartz
Logging: SLF4J (logging interface) + Logback (logging implementation)

Front end:

Thymeleaf
Bootstrap 4.x
Jquery
Ajax

🔨 Development Environment

Operating system: Windows 10
Build tool: Apache Maven
Integrated development tool: Intellij IDEA
Application server: Apache Tomcat
Interface test tool: Postman
Stress test tool: Apache JMeter
Version control tool: Git
Java version: 8

🎀 Interface Display

Home page:

The login page:

Post details page:

Personal Homepage:

Personal Message page:

Private Message Details page:

System Notification page:

Notice Details page:

Account setting page:

Data statistics page:

Search details page:

🎨 Feature List

registered
- The user is registered successfully, and the user information is stored in MySQL, but the user status is inactive
- An activation email is sent to the user, and the user clicks on the link to activate the account (Spring Mail)
Login | logout
- The login page is displayed, dynamically generates the verification code, and saves the verification code to the Redis for a short time (60 seconds).
- After the user successfully logs in (verify the user name, password, and verification code), the login certificate is generated, the status is set to valid, and the login certificate is saved to Redis. Note: Login credentials have a validity period and are checked for validity and expiration before any request is executed. As long as the user’s credentials are valid and remain valid for the duration of the validity period, the request holds the user information (using ThreadLocal to hold the user information).
- Check “Remember me” to extend the validity period of login credentials
- The user logged in successfully, and the user information was briefly stored in Redis (1 hour).
- The user logs out, sets the credential status to invalid and updates the login credential information of the user in Redis
Account Settings
- Modify the picture
  - Upload the profile picture file selected by the user to qiniu Cloud server
- Change the password
Post module
- Publish the post (filter sensitive words) and save it to MySQL
- Page displays all posts
  - The value can be displayed by Post Time
  - Support for “ranking by popularity” (Spring Quartz)
- View post details
- Spring Security + Thymeleaf Security
  - An unlogged user cannot post
  - The “moderator” can see the top and fine buttons of the post and perform corresponding operations
  - The “administrator” can see the delete button of the post and perform the corresponding action
  - “Ordinary users” cannot see the top, finish, delete buttons of posts, nor can they perform corresponding operations
Comments on the module
- Post comments on posts (filter sensitive words) and store them in MySQL
- Pagination of comments
- Post responses to comments (filter for sensitive words)
- Rights Management (Spring Security)
  - Users who are not logged in cannot comment
DMS module
- Send private messages (filter sensitive words)
- Direct messages list
  - Example Query the session list of the current user
  - Only one latest private message is displayed per session
  - Support paging display
- Personal details
  - Queries all private messages contained in a session
  - When accessing the private message details, the private message displayed is set to read
  - Support paging display
- Rights Management (Spring Security)
  - Users who are not logged in cannot use the private message function
The 404/500 exception is handled uniformly
- Normal request exception
- Asynchronous request exception
Unified Log Recording
Thumb up module
- Support to like posts, comments/replies
- Like the first time, unlike the second time
- The home page counts the number of likes of posts
- The details page counts the number of likes for posts and comments/replies
- The details page displays the “like” status of the currently logged in user (” like “indicates” like “).
- Count my likes
- Rights Management (Spring Security)
  - Users who are not logged in cannot use the like function
Focus on modules
- Focus on function
- Unfollow function
- Count the number of users’ attention and followers
- My follow list (to query the people a user follows), support paging
- My fan list (query a user’s fans), support paging
- Rights Management (Spring Security)
  - Users who are not logged in cannot use the follow function
System notification module
- Notification list
  - Display comments, likes, and follow notifications
- Inform the details
  - Pagination displays notifications that are contained within a certain type of topic
  - To enter a type of system notification details, set the status of all unread system notifications for that page to read
- Did not read the number
  - Displays the number of unread system notifications for each type, respectively
  - Displays the unread count of all system notifications
- Navigation bar shows unread number of all messages (unread private messages + unread system notifications)
- Rights Management (Spring Security)
  - Users who have not logged in cannot use the system notification function
Search module
- Publish event
  - When a post is published, it is asynchronously submitted to the Elasticsearch server via a message queue
  - When you add a comment to a post, the post is asynchronously submitted to the Elasticsearch server via a message queue
- Search service
  - Search for posts from the Elasticsearch server
  - Delete posts from Elasticsearch server (when posts are deleted from the database)
- Display search results
Website statistics (only for administrators)
- Independent Visitor UV
  - HyperLogLog in Redis
  - Supports one-day date query and interval date query
- Daily Active user DAU
  - Save to Redis Bitmap
  - Supports one-day date query and interval date query
- Rights Management (Spring Security)
  - Only administrators can view website statistics
Optimize site performance
- Cache the list of hot posts and the total number of all user posts using local cache Caffeine

🔐 to be implemented and optimized

The following are the problems I found in this project, but I have no idea to solve them. Let’s brainstorm, and everyone is welcome to propose A PR solution:

The registration module cannot jump to the operation prompt interface (there is no problem in local operation)
There is a Bug in the front display of the comment feature
Query my comments (incomplete)

The following are the functions that I think can be added to this project, and you are also welcome to point out what other functions can be added, or directly mention PR to achieve this function:

Forget password (send email to retrieve password)
Check my likes
Administrator to the post of the second click to cancel the top function
Administrator’s recovery function for deleted posts (deleted posts in this project are not deleted from the database, but their status is set to masking)

🌱 Running locally

If you need to deploy the project locally for testing, please prepare the following environment in advance:

Java 8
MySQL 5.7
Redis
2.13 2.7.0 Kafka
Elasticsearch 6.4.3

Then modify the information in the configuration file for your own local environment, the direct run is not running, and all the relevant private information I use XXXXXXX instead.

The configuration file to be modified is as follows:

1) application – develop. The properties:

MySQL
Spring Mail (SMTP service must be enabled for mailbox)
Kafka: consumer.group-id (see consumer.proerties in the Kafka installation package for this field, and you can modify it by yourself. After the modification, you need to restart Kafka)
Elasticsearch: cluster-name (see elasticsearch.yml in the Elasticsearch installation package)
Qiniuyun (need to create a new object storage space of Qiniuyun to store the uploaded profile picture)

2) logback – spring – develop. XML:

LOG_PATH: indicates the location where logs are stored

Each run needs to turn on:

MySQL
Redis
Elasticsearch
Kafka

In addition, you need the event to set up the database table, as detailed below.

📜 Database design

User user:

DROP TABLE IF EXISTS `user`;
SET character_set_client = utf8mb4 ;
CREATE TABLE `user` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(50) DEFAULT NULL,
  `password` varchar(50) DEFAULT NULL,
  `salt` varchar(50) DEFAULT NULL,
  `email` varchar(100) DEFAULT NULL,
  `type` int(11) DEFAULT NULL COMMENT '0- Regular user; 1- Super administrator; 2 - the moderator; ',
  `status` int(11) DEFAULT NULL COMMENT '0- Inactive; 1- Activated; ',
  `activation_code` varchar(100) DEFAULT NULL,
  `header_url` varchar(200) DEFAULT NULL,
  `create_time` timestamp NULL DEFAULT NULL.PRIMARY KEY (`id`),
  KEY `index_username` (`username`(20)),
  KEY `index_email` (`email`(20))
) ENGINE=InnoDB AUTO_INCREMENT=101 DEFAULT CHARSET=utf8;
Copy the code

Discuss_post:

DROP TABLE IF EXISTS `discuss_post`;
SET character_set_client = utf8mb4 ;
CREATE TABLE `discuss_post` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) DEFAULT NULL,
  `title` varchar(100) DEFAULT NULL,
  `content` text,
  `type` int(11) DEFAULT NULL COMMENT '0 - ordinary; 1 - placed at the top; ',
  `status` int(11) DEFAULT NULL COMMENT '0 - normal; 1 - essence; 2 - shield; ',
  `create_time` timestamp NULL DEFAULT NULL,
  `comment_count` int(11) DEFAULT NULL,
  `score` double DEFAULT NULL.PRIMARY KEY (`id`),
  KEY `index_user_id` (`user_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Copy the code

Comment:

CREATE TABLE `comment` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) DEFAULT NULL,
  `entity_type` int(11) DEFAULT NULL COMMENT 'Category of comment target: 1 post; 2 comments',
  `entity_id` int(11) DEFAULT NULL COMMENT 'Id of comment target',
  `target_id` int(11) DEFAULT NULL COMMENT 'Specify who to comment on',
  `content` text,
  `status` int(11) DEFAULT NULL COMMENT 'Status: 0 normal; 1 disabled ',
  `create_time` timestamp NULL DEFAULT NULL.PRIMARY KEY (`id`),
  KEY `index_user_id` (`user_id`),
  KEY `index_entity_id` (`entity_id`)
) ENGINE=InnoDB AUTO_INCREMENT=247 DEFAULT CHARSET=utf8;
Copy the code

Private message:

DROP TABLE IF EXISTS `message`;
SET character_set_client = utf8mb4 ;
CREATE TABLE `message` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `from_id` int(11) DEFAULT NULL,
  `to_id` int(11) DEFAULT NULL,
  `conversation_id` varchar(45) NOT NULL,
  `content` text,
  `status` int(11) DEFAULT NULL COMMENT '0 - read; 1 - read; 2 - removed; ',
  `create_time` timestamp NULL DEFAULT NULL.PRIMARY KEY (`id`),
  KEY `index_from_id` (`from_id`),
  KEY `index_to_id` (`to_id`),
  KEY `index_conversation_id` (`conversation_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Copy the code

🌌 Ideal deployment architecture

I only deployed one of each, and here’s the ideal deployment architecture:

🎯 Functional logic diagram

I’ve drawn some less serious pictures to help you clear your mind.

One-way green arrow:

Front-end template -> Controller: indicates that a hyperlink in the front-end template is handled by the Controller

Controller -> Front template: indicates that the Controller will pass data or jump to the front template

Bidirectional green arrow: indicates that parameters are passed or used between the Controller and the front template

One-way blue arrow: A -> B: method A calls method B

One-way red arrow: database or cache operation

registered

The user is registered successfully, and the user information is stored in MySQL, but the user status is inactive
An activation email is sent to the user, and the user clicks on the link to activate the account (Spring Mail)

Login | logout

The login page is displayed, dynamically generates the verification code, and saves the verification code to the Redis for a short time (60 seconds).
After the user logs in successfully (the user name, password, and verification code are verified), the login certificate is generated, the status is set to valid, and the login certificate is saved to Redis

Note: Login credentials have a validity period and are checked for validity and expiration before any request is executed. As long as the user’s credentials are valid and remain valid for the duration of the validity period, the request will hold the user information (using ThreadLocal to hold the user information).
Check “Remember me” to extend the validity period of login credentials
The user logged in successfully, and the user information was briefly stored in Redis (1 hour).
The user logs out, sets the credential status to invalid and updates the login credential information of the user in Redis

The following figure is the functional logic diagram of the login module, without using the authentication logic provided by Spring Security (I think this module is the most complex, in fact, many details have not been drawn in this diagram)

Page displays all posts

The value can be displayed by Post Time
Support for “ranking by popularity” (Spring Quartz)
The list of top posts and the total number of all posts are stored in a local cache, Caffeine (which uses the distributed timed task Spring Quartz to calculate the popularity/score of posts refreshed every once in a while – see below), while Caffeine doesn’t need to worry about data updates, it naturally updates the data it has automatically. Just give it an initialization method.)

Account Settings

Changing profile picture (Asynchronous Request)
- Upload the profile picture file selected by the user to qiniu Cloud server
Change the password

Only the modified head is drawn here:

Post (asynchronous request)

Displays comments and related information

The name of the front end of the comment section shows some defects, interested partners welcome to PR solution ~

One thing to note about the comment module is the design of the comment table. To fully understand the logic of this function, grasp the meaning of the fields in the comment table.

The target type of Comment (post, Comment) entityType and entityId and which user to Comment/reply to targetId is passed to the DiscussPostController by the front end

The details page of a post needs to encapsulate the following information:

Add comments (Transaction Management)

Private message list and details page

Send a private message (asynchronous request)

Like (asynchronous request)

The “like” information is stored in the Redis data structure set. Among them, the key named like: entity: entityType: entityId, value the thumb up user id. For example, key = like:entity:2:246 value = 11 indicates that user 11 has liked the comment of entity type 2. The comment id is 246

The number of likes of a user is stored in Redis with key like:user:userId and value is the number of likes of the user

The number of likes I got

Concern (asynchronous request)

If A follows B, then A is B’s Follower and B is A target Followee
The goal of attention can be users, posts, topics, etc., and these goals can be abstracted into entities during implementation (currently only focus on users).

Entities related to a user attention information is stored in the data structure zset Redis: key is followee: userId: entityType, the corresponding value is zset (entityId, now), to focus on time. For example, followee:111:3 corresponds to value (20, 2020-02-03-xxxx), indicating that the entity type of user 111 is 3, that is, person (user), the id of the post is 20, and the time of the post is 2020-02-03-xxxx

The same, will be an entity with fans of relevant information is also stored in the data structure zset Redis: key is follower: entityType: entityId, the corresponding value is zset (userId, now), to focus on time

Pay attention to the list

Sending System Notifications

Display system notifications

search

Similarly, the top and finish will also trigger the post event, which will not be drawn in the diagram.

Top finish delete (asynchronous request)

Website Statistics

Post Popularity calculation

Every time a post is liked, commented, or refined, the post information is stored in the cache Redis, and then the distributed timed task Spring Quartz is used to retrieve the posts from the cache at regular intervals to calculate the score.

Post score/heat calculation formula: score (heat) = weight + Posting distance days

// Calculate the weight
double w = (wonderful ? 75 : 0) + commentCount * 10 + likeCount * 2;
// Score = weight + Posting distance days
double score = Math.log10(Math.max(w, 1))
        + (post.getCreateTime().getTime() - epoch.getTime()) / (1000 * 3600 * 24);
Copy the code

📖

Want to realize this project from scratch or in-depth understanding of small partners, can scan the following TWO-DIMENSIONAL code to follow the public number “flying veal”, the first time to obtain the supporting tutorial, will not only explain in detail the project involved in each major technical points, will also summarize the relevant often meet questions, is still in the update.

And recommended my open source tutorial project “CS-Wiki”, Gitee recommended project, currently has 0.9 K star: committed to create a perfect Java backend knowledge system, not only to help you quickly and systematically prepare for the interview (autumn recruitment/social recruitment), but also to guide the direction of learning

👏 credit

The level of the blogger is limited, this project reference niuke.com – Java senior engineer course, thanks to the teacher and platform.