👍 for more exciting content, see [The original link】

Recommended: Java Technology Books – Awesome – Java-Books

From junior developers to experienced architects, these books should suffice

• The data structure

  • The queue
  • A collection of
  • Linked lists, arrays
  • Dictionaries, associative arrays
  • The stack

  • The tree

    • Binary tree
    • Complete binary trees
    • Balanced binary trees
    • Binary search tree (BST)
    • Red and black tree
    • B, B plus, B star tree
    • LSM tree
  • BitSet

• Commonly used algorithm

  • Sort, search algorithm

    • Selection sort
    • Bubble sort
    • Insertion sort
    • Quick sort
    • Merge sort
    • Hill sorting
    • Heap sort
    • Count sorting
    • Bucket sort
    • Radix sort
    • Binary search
    • Sort tools in Java
  • Bloon filter

  • String comparison

    • KMP algorithm
  • Depth first, breadth first
  • Greedy algorithm
  • Backtracking algorithm
  • Pruning algorithm
  • Dynamic programming
  • Naive Bayes
  • Recommendation algorithm
  • Minimum spanning tree algorithm
  • Shortest path algorithm

• concurrent

  • Java concurrency
  • multithreading
  • Thread safety

  • Consistency, transaction

    • Transaction ACID feature
    • The isolation level of the transaction
    • MVCC

  • The lock

    • Lock and synchronization classes in Java
    • Fair lock & non-fair lock
    • Pessimistic locking
    • Optimistic Lock & CAS
    • ABA problem
    • CopyOnWrite container
    • RingBuffer
    • Reentrant lock & non-reentrant lock
    • Mutex & Shared Lock
    • A deadlock

• The operating system

  • Principles of computer

  • CPU

    • Multistage cache
  • process
  • thread
  • coroutines
  • Linux

• Design patterns

  • Six principles of design patterns
  • 23 common design patterns
  • Application scenarios
  • The singleton pattern
  • Chain of responsibility model
  • MVC
  • IOC
  • AOP
  • UML

  • Micro-service thought

    • Conway’s law

• Operations & Statistics & Technical Support

  • Routine monitoring
  • APM
  • Statistical analysis

  • Continuous Integration (CI/CD)

    • Jenkins
    • Environmental separation

  • Automatic operation and maintenance

    • Ansible
    • puppet
    • chef

  • test

    • TDD theory
    • Unit testing
    • Pressure test
    • Full link compression test
    • A/B, gray, blue and green test

  • virtualization

    • KVM
    • Xen
    • OpenVZ

  • Container technology

    • Docker

  • The cloud

    • OpenStack
  • DevOps
  • Document management

• The middleware

  • Web Server

    • Nginx
    • OpenResty
    • Tengine
    • Apache Httpd

    • Tomcat

      • Architecture principles
      • The tuning scheme
    • Jetty

  • The cache

    • The local cache
  • Client-side caching

  • Server-side caching

    • Web caching
    • Memcached

    • Redis

      • architecture
      • Recovery strategy
    • Tair

  • The message queue

    • The message bus
    • Order of messages
    • RabbitMQ
    • RocketMQ
    • ActiveMQ
    • Kafka
    • Redis newsfeed
    • ZeroMQ

  • Timing schedule

    • Single machine timing scheduling
    • Distributed Scheduling

  • RPC

    • Dubbo
    • Thrift
    • gRPC

  • Database middleware

    • Sharding Jdbc

  • Logging system

    • Log collection
  • Configuration center
  • API gateway

• network

  • agreement

    • OSI seven layer protocol
    • TCP/IP
    • HTTP
    • HTTP2.0
    • HTTPS

  • A network model

    • Epoll
    • Java NIO
    • kqueue
  • Connections and short connections
  • The framework
  • Zero copy

  • Serialization (binary protocol)

    • Hessian
    • Protobuf

• The database

  • The basic theory

    • Three paradigms for relational database design

  • MySQL

    • The principle of
    • InnoDB
    • To optimize the

    • The index

      • Clustered index, non-clustered index
      • The composite index
      • Adaptive Hash Index (AHI)
    • explain

  • NoSQL

    • MongoDB
    • Hbase

• Search engine

  • Principles of search engines
  • Lucene
  • Elasticsearch
  • Solr
  • sphinx

• performance

  • Performance optimization methodology
  • Capacity assessment
  • CDN network
  • The connection pool
  • Performance tuning

• Big data

  • Flow calculation

    • Storm
    • Flink
    • Kafka Stream
    • Application scenarios

  • Hadoop

    • HDFS
    • MapReduce
    • Yarn
  • Spark

• security

  • Web security

    • XSS
    • CSRF
    • SQL injection
    • Hash Dos
    • Script injection
    • Vulnerability scanning tool
    • Verification code
  • To prevent DDoS
  • User privacy information protection
  • Serialization vulnerability

  • Encryption to decrypt

    • Symmetric encryption
    • The hash algorithm
    • Asymmetric encryption
  • Server security

  • Data security

    • The data backup

  • Network isolation

    • Internal and external network separation
    • Log on to the jumper

  • Authorization, authentication

    • RBAC
    • OAuth2.0
    • OIDC
    • SAML
    • Two-factor certification (2FA)
    • Single Sign-On (SSO)

• Common open source frameworks

  • Open source licenses

  • The logging framework

    • Log4j, Log4j2
    • Logback
  • ORM
  • Network framework

  • Web framework

    • The family of the Spring
  • Tool framework

• Distributed design

  • Extensibility design

  • Stability & high availability

    • Hardware load balancing
    • Software load balancing
    • Current limiting
    • Application layer disaster tolerance
    • Disaster tolerance across the machine room
    • Disaster recovery drill process
    • A smooth start

  • Database expansion

    • Read-write separation mode
    • Fragmentation patterns

  • Service governance

    • Service registration and discovery
    • Service routing control

  • Distributed consistency

    • Cap and BASE theory
    • A distributed lock

    • Distributed consistency algorithm

      • PAXOS
      • Zab
      • Raft
      • Gossip
      • Two-phase commit, multi-phase commit
    • Power etc.
    • Distributed consistent scheme
    • Distributed Leader node election
    • TCC(Try/Confirm/Cancel) flexible transactions
  • Distributed file system

  • Unique ID generation

    • Globally unique ID
  • Consistent Hash algorithm

• Design Idea & Development Pattern

  • DDD(Domain-Driven Design – Domain Driven Design)

    • Command Query Separation of Responsibilities (CQRS)
    • Anemia, congestion model
  • The Actor model

  • Reactive programming

    • Reactor
    • RxJava
    • Vert.x
  • DODAF2.0
  • Serverless
  • Service Mesh

• The project management

  • Architecture review
  • refactoring
  • Code specification
  • Code Review
  • RUP
  • Kanban management
  • SCRUM
  • Agile development
  • Extreme Programming (XP)
  • Pair programming
  • PDCA cycle quality management
  • FMEA management mode
• General business terms
• Technology trends

• Policies and regulations

  • The law

    • Strictly abide by Article 253 of the Criminal Law
• Quality of Architect

• Team management

  • recruitment

• information

  • Industry information
  • List of Public Accounts

  • blog

    • Team blog
    • Personal blog
  • Integrated portal, community
  • Q&A and discussion communities
  • Industry data analysis
  • The special website
  • Other classes

  • Recommended reference book

    • Online e-books

    • Paper books

      • development
      • Architectural aspects
      • Technical Management
      • The basic theory
      • tools
      • Big data

• Technical resources

  • Open source resources
  • Manuals, documentation, tutorials
  • Online classroom
  • Meetings and activities
  • Commonly used APP
  • Looking for a job
  • tool
  • Managed code
  • File service

  • Integrated Cloud Service Provider

    • VPS

(Toc generated bysimple-php-github-toc ）

The data structure

The queue

• “Java Queues — Detailed Analysis of Queues”

  • Non-blocking Queue: ConcurrentLinkedQueue(unbounded thread-safe), using the CAS mechanism (CompareAndSwapObject atomic operation).
  • BlockingQueue: ArrayBlockingQueue(bounded), LinkedBlockingQueue (unbounded), DelayQueue, PriorityBlockingQueue, using locking mechanism; Use the ReentrantLock lock.
• LinkedList, ConcurrentLinkedQueue, LinkedBlockingQueue

A collection of

• “Java Set Collection”

Linked lists, arrays

• What is a List?

Dictionaries, associative arrays

• Java Map – Usage, Traversal, Sorting, Common API, etc.

The stack

• Design and Implementation of Java Data Structures and Algorithms Stack
• “Java Stack Class”

• “Detailed implementation analysis of Java Stack”

  • Stack is thread-safe.
  • The internal use of an array to store data, when insufficient to double.

The tree

Binary tree

Each node has a maximum of two leaf nodes.

• Binary Tree

Complete binary trees

• Complete Binary Tree

  • Leaf nodes can only appear in the lowest layer and the second layer, and the nodes of the lowest layer are all concentrated in the left of the binary tree.

Balanced binary trees

The absolute value of the height difference between the left and right subtrees is no more than 1, and both the left and right subtrees are a balanced binary tree.

• Discussion on Data Structure – Balanced Binary Tree
• A Brief Introduction to Algorithms and Data Structures: 2-3 Trees of Eight Balance Search Trees

Binary search tree (BST)

Binary Search Tree Ordered Binary Tree sorted Binary Tree sorted Binary Tree sorted Binary Tree sorted Binary Tree sorted

• A Brief Discussion on Algorithms and Data Structures: Seven Binary Search Trees

Red and black tree

• The Easiest to Understand Red-Black Trees

  • After the addition phase, it rotates left or right to reach equilibrium again.
• A Brief Discussion on Algorithms and Data Structures: Red-Black Trees of Nine Balanced Search Trees

B, B plus, B star tree

MySQL is based on B+ tree clustered index to organize tables

• B- Tree, B+ Tree, B* Tree in detail

• Comparison of Advantages and Disadvantages of B-Tree, B+ Tree and B* Tree

  • Compared with B-tree, the leaf node linked list structure of B+ tree is easier to scan the database and search the range.

  LSM tree

LSM (Log-Structured Merge-Trees), in contrast to B+ Trees, trades partial read performance for write performance (through batch writing), achieving a read-write balance. HBase, LevelDB, TAIR (Long DB) and NESSDB adopt the structure of LSM tree. LSM allows for fast indexing.

• LSM Tree VS B+ Tree

  • B+ tree read performance is good, but due to the need for an ordered structure, when the keys are scattered, the disk seeks frequently, resulting in poor write performance.
  • LSM is to split a big tree into N small trees, write to memory first (no trace problem, high performance), in memory to build an ordered small tree (ordered tree), as the small tree gets bigger and bigger, the memory of the small tree will be flushed to the disk. When reading, we do not know which tree the data is in, so we must traversal (binary search) all the trees, but within each tree the data is ordered.

• LSM (Log-Structured Merge Tree) Storage Engine

  • At the extreme, the LSM tree-based implementation of HBase has an order of magnitude higher write performance and an order of magnitude lower read performance than MySQL.
  • Optimization method: Bloom filter replaces binary search; Compact Minimal Tree to improve query performance.
  • HDFS does not support UPDATE operations, so HBase does not do an overall flush instead of a merge update. Small trees flushed to disk will periodically merge into one big tree.

BitSet

It is often used for reload checking of large scale data.

• “Java Bitset class”
• “Java BitSet”

Commonly used algorithm

• Common Sorting Algorithms and Their Corresponding Time and Space Complexity

Sort, search algorithm

• Common Sorting Algorithms and Their Corresponding Time and Space Complexity

Selection sort

• “SelectionSort: Classical Algorithms in Java”

  • Each pass selects the smallest element from the records to be sorted, and the order is placed at the end of the sorted sequence until all records are sorted.

Bubble sort

• Two Ways of Writing Bubble Sort

  • Adjacent elements are swapped back and forth, ranking the largest last.
  • Time complexity O(n ^ 2)

Insertion sort

• Insertion Sort Summary of Sort Algorithms

Quick sort

• Sitting on the Toilet Looking at Algorithms: Quick Sort

  • One side is larger or smaller than the other.

  Merge sort

• Graphic Sort Algorithm (IV) Merge Sort

  • Divide and conquer, sort into smaller pieces, merge (rebuild a new space to copy).

Hill sorting

TODO

Heap sort

• Graphic Sort Algorithm (III) Heap Sort

  • Sorting is the process of building the largest heap, the largest heap: the value of each node is greater than or equal to the value of its left and right children, and the top element of the heap is the maximum.

Count sorting

• Count Sort and Bucket Sort

  • It’s similar to the bucket sorting process, but the difference is the number of buckets.

Bucket sort

• [Aha! Algorithm] The Fastest and Simplest Sort — Bucket Sort

• Sort algorithm (3) : counting sort and bucket sort

  • Bucket sort divides the interval [0,1) into n subintervals of the same size, which are called buckets.
  • Sort each bucket individually, and then iterate over each bucket.

Radix sort

In the ones place, the tens place, the hundreds place… Let’s do it one by one.

• Sorting Algorithms Series: Radix Sorting
• Radix Sort

Binary search

• “Binary Search (Java Implementation)”

  • Require that the sequence to be found be ordered.
  • Order logN.

• “Java implementation binary search – two ways”

  • While + recursion.

  Sort tools in Java

• Arrays.Sort and Collections.Sort Implementation Principle Analysis

  • The Collections.sort algorithm calls merge sort.
  • Arrays.sort() uses two sort algorithms — quicksort for basic types of data and merge sort for Arrays of objects.

Bloon filter

It is often used for reordering big data, such as email, URL, etc. Core Principle: Each piece of data is computed to produce a fingerprint (one byte or more bytes, but certainly much less than the original data), each of which is obtained by random calculation, in the fingerprint mapping into a large bitwise storage space. Note: there will be a certain error rate. Advantages: high efficiency in both space and time. Disadvantages: As the number of elements stored increases, the error rate increases.

• Bloon Filters — Spatial Efficient Data Structures
• “Deduplication of Large Data: Bitmaps and Bloom Filters”

• Realization of Redis-based Bloon Filter

  • Redis-based Bitmap data structure.

• Web Crawlers: The Use of BloomFilter for URL De-Rendering Strategies

  • Use the BitSet class in Java and the weighted and hash algorithms.

String comparison

KMP algorithm

KMP: The core principle of Knuth-Morris-Pratt algorithm (KMP for short) is to use a “partial match table” to skip the elements that have already been matched.

• KMP Algorithm for String Matching

Depth first, breadth first

• Breadth First Search BFS and Depth First Search DFS

Greedy algorithm

• Algorithms: The Foundations of Greedy Algorithms
• Common Algorithms and Problem Scenarios — Greedy Algorithms

Backtracking algorithm

• “Five Common Algorithms Four: Backtracking”

Pruning algorithm

• “α-β pruning algorithm”

Dynamic programming

• Detailed Dynamic Programming — Zou Bo’s Discussion on Dynamic Programming
• Personal Understanding of Dynamic Programming Algorithms

Naive Bayes

• Understanding Naive Bayesian Classification Algorithms

  • P(B|A)=P(A|B)P(B)/P(A)
• Bayesian Inference and Its Application on the Internet 1
• Bayesian Inference and Its Application on the Internet 2

Recommendation algorithm

• Review of Recommendation Algorithms
• Introduction to Top 10 Open Source Recommendation Systems

Minimum spanning tree algorithm

• Introduction to Algorithms — Minimum Spanning Tree (Kruskal and Prim Algorithms)

Shortest path algorithm

• “Dijkstra algorithm detail”

concurrent

Java concurrency

• Java Concurrency Knowledge Collection
• Java Concurrency Knowledge Graph

multithreading

• Summary of 40 Java Multithreading Problems

Thread safety

• Concurrent Programming in Java — An Introduction to Thread Safety and Resolution Mechanisms

Consistency, transaction

Transaction ACID feature

• Database transaction ACID feature

The isolation level of the transaction

• Uncommitted reads: A transaction can read uncommitted data from another transaction and is prone to dirty reads.
• Read commit: A transaction waits for another transaction to commit before it can read data, but it will not be able to read repeatedly (multiple reads of data are inconsistent), and the UPDATE operation will occur during the read process. (The default level of most databases is RC, such as SQL Server, Oracle), which cannot be changed when read.
• Readable: The same transaction can be read each time, the same data is obtained, but does not guarantee that the original data is updated by other transactions (magic), MySQL InnoDB is this level.
• Serialization: Everything is processed serially (sacrificing efficiency)
• Four Isolation Levels for Understanding Transactions
• Four characteristics of database transactions and transaction isolation levels

• InnoDB’s Phantom Problem in MySQL

  • The example of phantom reading is very clear.
  • Through the SELECT… Solve FOR UPDATE

• Understanding MySQL and InnoDB in an article

  • Illustrate dirty reading, non-repeatable reading, phantom reading problems.

MVCC

• InnoDB MVCC: InnoDB MVCC: InnoDB MVCC: InnoDB MVCC

  • InnoDB uses MVCC for Repeatable Read isolation level.
  • MVCC can cause phantom problems (exceptions when updating).

• MySQL MVCC implementation mechanism is easy to understand

  • MVCC control is achieved by hiding the version columns, one for creation, one for deletion, here
  • Operate only on rows smaller (or equal to) than the current version at a time.

The lock

Lock and synchronization classes in Java

• Lock Classification in Java

  • These include synchronized, ReentrantLock, and ReadWriteLock.
• “Java Concurrency AQS detail”

• Semaphore in Java

  • There is quantity control
  • Acquire, do not block; ‘Release’ is used to release.

• Mutex vs. Semaphore in Java Development

  • In short, Mutex is exclusive and only one object can access resources. Semaphore is also exclusive, but can define multiple objects that can access resources.

Fair lock & non-fair lock

The function of a fair lock is to execute it strictly in the order in which the thread starts, and not to allow other threads to jump the queue. Non-fair locks allow queue-jumping.

• Fair and Unfair Lock

  • By default, both ReentrantLock and synchronized are unfair locks. ReentrantLock can be set as a fair lock.

Pessimistic locking

Pessimistic locks, if used improperly (too many locks), can cause large waits for the service. Optimistic locking + retry is preferred.

• [MySQL] Pessimistic Locking & Optimistic Locking

  • Optimistic locking mode: version number + retry mode
  • Pessimistic lock: select… For update performs row locking (unreadable, unwritable, share lock readable, unwritable).

• SQL > select… for update; SQL > select… for update;

  • MySQL’s InnoDB storage engine practice lock is a lock row, but it is internal lock index.
  • Locking different index conditions for the same data can cause a deadlock.
• Deadlock Causes and Solutions for Concurrency in MySQL

Optimistic Lock & CAS

• An Implementation Method of Optimistic Lock — CAS

  • Similar to MySQL optimistic locking, but by comparing it to the original value.

ABA problem

Due to high concurrency, under CAS, this A may not be that A after the update. This can be solved by version number, similar to the optimistic locking mentioned in MySQL above.

• Java CAS and ABA Issues

• ABA Problems and Avoidance in Java

  • AtomicStampedReference and AtomicStampedReference.

CopyOnWrite container

Concurrent reads can be made to the CopyonWrite container without locking it. The CopyonWrite Concurrency Container is used in concurrent scenarios where there are too many reads and too few writes. For example, whitelist, blacklist, and commodity class access and update scenarios are not suitable for scenarios requiring strong data consistency.

• Copy-on-write Map implementation in Java

  • Realize read-write separation, with reads occurring on the original data and writes occurring on the copy.
  • Consistency is achieved through final consistency without locking.
• “Copy-on-write Containers in Concurrency — Java”

RingBuffer

• Implementation of thread-safe lockless RingBuffer [one reader thread, one writer thread]

Reentrant lock & non-reentrant lock

• Reentrant and Non-reentrant Lock

  • Reentrant and non-reentrant locks are illustrated with simple code examples.
  • A reentrant lock means that the same thread can reacquire a previously acquired lock.
  • Reentrant locks allow users to avoid deadlocks.
  • In Java reentrant lock: synchronized and Java. Util. Concurrent. The locks. Already

• “Summary of ReentrantLock Reentrant (Synchronized)”

  • Synchronized is easy to use, and the compiler adds locks that are not fair.
  • ReentrantLock is flexible and the fairness of the lock can be customized.
  • Synchronized is recommended for the same locking scenario.

Mutex & Shared Lock

Mutex: Only one thread can acquire the lock at a time. For example, ReentrantLock is a mutex, and the write lock in ReadWriteLock is a mutex. Shared lock: can have multiple threads or locks at the same time. For example, Semaphore and countDownLatch are shared locks, and read locks in ReadWriteLock are shared locks.

• ReadWriteLock Scenario Application

A deadlock

• Reasonable Explanation of the Four Necessary Conditions of “Deadlock”

  • Mutual Exclusive, Holding, Inalienable, Loop Wait.

• How does Java view deadlock?

  • JConsole can recognize deadlocks.

• Java Multithreading Series: Deadlocks and Detection

  • Jstack can show deadlocks.

The operating system

Principles of computer

• Operating System Fundamentals — Principles, Types, and Structures of Operating Systems

CPU

Multistage cache

A typical CPU has a three-level cache, where the closer it is to the core, the faster it is and the less space it has. L1 generally 32K, L2 generally 256K, L3 generally 12M. The memory speed takes 200 CPU cycles, and the CPU cache takes 1 CPU cycle.

• “Understanding CPU Caching and Pseudo Sharing from a Java Perspective”

process

TODO

thread

• Thread life cycle and state transitions in detail

coroutines

• “Ending Python coroutines —- Implementing the yield to actor model”

  • Thread scheduling is responsible by the operating system, coroutine scheduling is responsible by the program itself
  • Compared to threads, coroutines reduce unnecessary operating system switching.
  • In practice, it makes more sense to switch when an IO operation is encountered (because the IO operation does not take up CPU), and if no IO operation is encountered, switch by time slice.

Linux

• The Linux Command Collection

Design patterns

Six principles of design patterns

• Six Principles of Design Patterns

  • Open Closed Principle: Open for extensions, closed for modifications, use abstract classes and interfaces.
  • The Richter Replacement Principle: Base classes can be replaced by subclasses, using abstract class inheritance, not using concrete class inheritance.
  • Rule of Dependency Inversion: Rely on abstractions, not concrete, program to interfaces, not implementations.
  • The principle of interface isolation: it is better to use multiple isolated interfaces than a single interface. Establish the smallest interface.
  • Demeter’s Law: A software entity should interact as little as possible with other entities, establishing relationships through intermediate classes.
  • Rule of composition reuse: Use composition/aggregation instead of inheritance whenever possible.

23 common design patterns

• Design Patterns
• Complete Analysis of 23 Design Patterns
• Design Pattern Class Diagrams and Examples

Application scenarios

• “Design Patterns in the JDK”

  • Structural mode:

    • Adapters: Used to convert one interface to another, such as java.util.arrays# asList().
    • Bridging pattern: This pattern decouples the implementation of abstraction and abstraction operations so that the abstraction and implementation can change independently, as in JDBC;
    • Composition pattern: Makes a single object and a composition of objects appear equal to the client. In other words, a method of a certain type also takes its own type as an argument, such as Map.PutAll, List.AddAll, and Set.AddAll.
    • Decorator pattern: dynamic attaches additional functionality to an object, it is also a subclass of an alternative way, such as Java. Util. Collections# checkedList Map for | | Set | SortedSet | SortedMap.
    • Entrance pattern: Use caching to speed up access time for a large number of small objects, such as valueOf(int).
    • Proxy pattern: The Proxy pattern replaces a complex or time consuming object with a simple one, such as java.lang.reflect.proxy

  • Create mode:

    • Abstract Factory pattern: The Abstract Factory pattern provides a protocol to generate a series of related or independent objects without specifying the type of the specific object, such as java.util.calendar #getInstance().
    • Builder: Defines a new class to build an instance of another class to simplify the creation of complex objects such as java.lang.StringBuilder#append().
    • Factory method: One method that returns a concrete object, not multiple, such as java.lang.object# toString(), java.lang.class# newInstance().
    • Prototype pattern: Enables instances of a class to make copies of themselves, such as java.lang.object# clone().
    • Singleton pattern: Only one instance globally, such as java.lang.runtime #getRuntime().

  • Behavior pattern:

    • Chain of Responsibility pattern: Decoupling objects by passing requests from one object to the next object in the chain until the request is processed. Such as javax.mail. Servlet. Filter# doFilter ().
    • Command pattern: Encapsulates operations into objects for storage, delivery, and return, such as java.lang.Runnable.
    • Interpreter pattern: Defines the syntax of a language and then parses the corresponding syntax statements, such as, java.text.format, java.text.normalizer.
    • Iterator pattern: Provides a consistent way to access objects in a collection sequentially, such as java.util.Iterator.
    • Mediator pattern: By using an intermediate object for message distribution and reducing direct dependencies between classes, java.lang.reflect.method# invoke().
    • Empty object patterns: such as java.util.collections# emptyList().
    • Observer pattern: This gives an object the flexibility to send messages to objects of interest, such as java.util.EventListener.
    • Template method pattern: Allows subclasses to override part of a method rather than the whole thing, such as java.util.collections #sort().
• Spring- A Summary of Design Patterns Intended
• Design Patterns Used by MyBatis

The singleton pattern

• Three Implementations of the Singleton Pattern and Their Advantages and Disadvantages

• “Singleton Pattern — Reflection — Preventing Serialization from Breaking Singleton Pattern”

  • Use enumerated types.

Chain of responsibility model

TODO

MVC

• The MVC Pattern

  • Model – View – Controller

IOC

• Understanding the IOC

• Understanding and Interpretation of the IOC

  • Forward control: Traditional through new. Reverse control, injecting objects through the container.
  • Function: For module decoupling.
  • DI: Dependency Injection (DI) is an Injection that only cares about resource usage, not resource source.

AOP

• Easy to understand AOP(Facet-Oriented Programming)
• Spring AOP in detail

• “The Principle of Spring AOP Implementation”

  • Spring AOP uses dynamic proxies in two main ways: JDK dynamic proxies and CGLIB dynamic proxies.

• Spring AOP Implementation Principle and CGLIB Application

  • The Spring AOP framework treats AOP proxy classes as follows: If the target object’s implementation class implements the interface, Spring AOP will use JDK dynamic proxies to generate AOP proxy classes. If the target object’s implementation class does not implement an interface, Spring AOP will use CGLIB to generate the AOP proxy class

UML

• UML Tutorial

Micro-service thought

• Design of Micro Service Architecture
• Technical Stack Selection Manual for Micro Service Architecture

Conway’s law

• The Theoretical Basis of Microservice Architecture – Conway’s Law

  • Law 1: The way an organization communicates is expressed through the system design, which means that the layout of the architecture and the organization structure will be similar.
  • Law 2: You can’t do a thing perfectly if you have more time, but there’s always time to do it. You can’t eat fat at a stretch. You can fix it first.
  • Law 3: There are potential homomorphic properties between linear systems and linear organizational structures. As an independent and autonomous subsystem, reduce communication costs.
  • Law 4: Large systems are more likely to be broken down than small systems. Divide, divide and rule.
• 20 lectures on the core of micro-service architecture

Operations & Statistics & Technical Support

Routine monitoring

• “The Road to Cultivation of Tencent Business System Monitoring”

  • Monitoring methods: active, passive and bypass (such as public opinion monitoring)
  • Monitoring types: basic monitoring, server monitoring, client monitoring, monitoring, client monitoring
  • Monitoring target: complete, block, accurate
  • Core metrics: request volume, success rate, time consuming

• Open source or commercial? A review of the top ten cloud operation and maintenance monitoring tools

  • Zabbix, Nagios, Ganglia, Zenoss, Open-Falcon, Monitoring Bao, 360 website service monitoring, Aliyun monitoring, Baidu cloud observation, Bee website monitoring, etc.
• Monitoring and Alarm System Construction and Secondary Development Experience

Command line monitoring tool

• “Common Command Line Monitoring Tools”

  • TOP, SAR, TSAR, NLOAD
• 20 Command Line Tools to Monitor Linux System Performance
• JVM performance tuning monitoring tools JPS, jstack, jmap, jhat, jstat, hprof

APM

APM — Application Performance Management

• Dapper, A Tracking System for Large-Scale Distributed Systems
• CNCF OpenTracing, Chinese Version

• Major open source software in alphabetical order

  • Apache SkyWalking
  • CAT
  • CNCF jaeger
  • Pinpoint
  • Zipkin

• Open Source APM Technology Selection and Practice

  • Mainly based on Google’s Dapper (Tracking System for Large-Scale Distributed Systems) idea.

Statistical analysis

• The Basis of Traffic Statistics: Buried Point

  • Common indicators: visits and visitors, length of stay, jump out rate, exit rate, conversion rate, participation

• “Common Statistical Tools, Buried Point Objectives and Buried Point Contents of APP Buried Point”

  • Third-party statistics: Youmeng, Baidu Mobile, Rubik’s Cube, App Annie, Talking Data, Shence Data, etc.

• Meituan Review Practice of Front End Traceless Buried Point

  • The so-called non-trace means that the acquisition node is configured through a visual tool, and the configuration is automatically analyzed and the buried point data is reported at the front end, rather than hard-coded.

Continuous Integration (CI/CD)

• What is Continuous Integration?
• 8 Popular Continuous Integration Tools

Jenkins

• Continuous Integration with Jenkins

Environmental separation

Separation of development, test, and build environments.

• Basic Understanding and Area of Development Environment, Production Environment and Test Environment

Automatic operation and maintenance

Ansible

• Ansible Authoritative Guide to Chinese
• Ansible Basic Configuration and Practical Cases for Enterprise Projects

puppet

• Puppet Detail: Automated Operation and Maintenance Tools

chef

• Installation and Use of Chef

test

TDD theory

• Deep Reading – TDD (Test Driven Development)

  • Coding functional code based on test cases, the core practice of XP (Extreme Programming).
  • Benefits: focus on one point at a time, reduce the burden of thinking; Meet changing requirements or improve code design; Clarify requirements in advance; Fast feedback;

Unit testing

• JUnit Part of Java Unit Testing

• JUnit 4 vs. TestNG

  • TestNG overrides JUnit functionality and is suitable for more complex scenarios.

• “Major Test Function Points of Unit Testing”

  • Module interface test, local data structure test, path test, error handling test, boundary condition test.

Pressure test

• Apache AB Testing Guide
• Stress Test and Optimization Scheme of Large-scale Website
• Top 10 Recommended Pressure/Load/Performance Testing Tools
• Analysis on the Application of the Real Flow Pressure Measurement Tool TCPCOPY
• Ngrinder Easy to Use Tutorial

Full link compression test

• JD.com 618: Upgrade the Full-Link Test Scheme to Build ForceBot for Military Exercises
• Exploration and Practice of Ele. me Full Link Pressure Test
• The Solution of | Full Link Test in Four Languages and Eight Frameworks
• “Full Link Pressure Test Experience”

A/B, gray, blue and green test

• The technology dry | AB test and gray level to explore and practice”
• “Nginx grayscale publishing by IP”
• Blue and Green Deployment, A/B Testing, and Grayscale Release

virtualization

• “Comparison of Advantages and Disadvantages of OpenVZ, Xen and KVM of the three VPS Virtual Technologies”

KVM

• “KVM detail, too detailed too deep, classic”
• 【 Text and text 】KVM virtual machine installation details

Xen

• “Xen Virtualization Fundamentals in detail”

OpenVZ

• Quick-Start Guide to OpenVZ Open Source Linux Container

Container technology

Docker

• A few diagrams to help you understand the basics of Docker and a quick start
• Docker Core Technology and Implementation Principle
• Docker Tutorial

The cloud

OpenStack

• OpenStack Framework Knowledge Combing

DevOps

• What the hell is DevOps in a Minute?
• DevOps in detail

Document management

• Confluence- Charged document management system
• GitLab?
• Wiki

The middleware

Web Server

Nginx

• “Basic Learning for NGNix – Comparison of Multiple Processes and Apache”

  • Nginx achieves high concurrency through an asynchronous non-blocking event handling mechanism. Apache hogs one thread per request, which is a drain on system resources.
  • Event driven works well for IO intensive services (NGINX), and multiple processes or threads work well for CPU intensive services (Apache), so NGINX works well for reverse proxies, not for Web servers.

• “Nginx vs. Apache and the pros and cons”

  • Nginx is only suitable for static and reverse proxies, not for dynamic requests.

OpenResty

• The official website

• A Brief Talk about OpenResty

  • Development on Nginx is possible through the Lua module.
• Agentzh’s Nginx tutorial

Tengine

• The official website

Apache Httpd

• The official website

Tomcat

Architecture principles

• Tomcat Principles and Request Process
• “Tomcat Server Principles in detail”
• Tomcat System Architecture and Design Patterns, Part 1: How It Works
• Tomcat Architecture in Four Graphics

• JBoss vs. Tomcat: Choosing A Java Application Server

  • Tomcat is a lightweight Serverlet container that does not implement all of the JEE features (such as persistence and transaction processing), but can be replaced by other components, such as Spring.
  • JBoss has achieved all the JEE features, the software is open source and free, the documentation is charged.

The tuning scheme

• Tomcat Tuning Solution

  • Start NIO mode (or APR); Adjust thread pool; Disable the AJP connector (Nginx+ Tomcat architecture does not require AJP);
• Tomcat HTTP Protocol and AJP Protocol

• Comparison and Analysis of AJP and HTTP

  • The AJP protocol (port 8009) is used to reduce the number of connections (front end) to the front-end Server (such as Apache, which needs to support the AJP protocol) to improve performance through long connections.
  • When concurrency is high, AJP is superior to HTTP.

Jetty

• How Jetty works and how it compares to Tomcat

• Jetty vs. Tomcat Advantages

  • Architecture comparison :Jetty’s architecture is simpler than Tomcat’s.
  • Performance comparison: There is little difference between Jetty and Tomcat in terms of performance. Jetty’s default NIO terminal is more favorable in handling I/O requests, while Tomcat defaults to BIO for I/O requests. Tomcat is suitable for handling a few very busy links and has poor performance when handling static resources.
  • Other aspects: Jetty applications are faster, easy to modify, and has better support for the new Servlet specification; Tomcat has more comprehensive support for JEE and servlets.

The cache

• Cache invalidation strategy (the difference between FIFO, LRU and LFU algorithms)

The local cache

• HashMap Local Cache

• Ehcache Local Cache

  • In-heap, out-of-heap, and disk-level caches.
  • Can be set according to the cache space capacity.
  • Expiration strategy according to time, number and so on.

• The Guava Cache”

  • Simple and lightweight, no out of heap, disk cache.
• Nginx Local Cache
• “PageSpeed — Lazy Tool, Server-Side Acceleration”

Client-side caching

• Browser-Side Caching

  • The main use of Cache-Control parameters.
• “H5 and Mobile WebView Cache Mechanism Analysis and Practice”

Server-side caching

Web caching

• nuster – nuster cache
• varnish – varnish cache
• squid – squid cache

Memcached

• Memcached Tutorial

• Deep Understanding of the Principles of Memcached

  • Adopting multiplexing technology to improve concurrency.
  • Memcached allocates memory space to slab. The default is 1MB. After assigning a Slab, divide the Slab into equally sized chunks, which are the memory space used to cache the records. By default, the chunk size increases by a factor of 1.25. Advantages are less frequent application of memory, improve IO efficiency, the disadvantage is that there will be a certain amount of memory waste.
• “How Memcached Software Works”
• Memcache Technology Sharing: Introduction, Use, Storage, Algorithm, Optimization, Hit Ratio

• “The difference between add, set, and replace in memcache”

  • The difference is that the return value is true or false when the key exists or does not exist.
• “The Anatomy of Memcached”

Redis

• Redis Tutorials

• Redis Fundamentals

  • Use ZipList to store linked lists. ZipList is a kind of compressed linked list. The advantage of ZipList is that it saves more memory space because the contents are stored in contiguous memory areas.
  • Skiplist (skiplist) is used to store ordered collection objects, search from a high Level first, the time complexity is similar to the red-black tree, easy to implement, no lock, good concurrency.

• Redis Persistence

  • RDB mode: Backup snapshots regularly, often used for disaster recovery. Advantages: Backup by fork without affecting the main process. RDB can restore large data sets faster than AOF. Disadvantages: Loss of data.
  • AOF mode: save operation log mode. Advantages: less data loss during recovery, disadvantages: large file, slow recovery.
  • You can use a combination of both.
• Distributed Caching – Sequence 3- Atomic Operations and CAS Optimistic Locking

architecture

• Redis Single Thread Architecture

Recovery strategy

• Redis’s Recycling Strategy

Tair

• The official website
• Tair vs. Redis
• Features: You can configure the number of backup nodes and synchronize to the backup nodes asynchronously
• Consistent Hash algorithm.
• Architecture: Similar to the design idea of Hadoop, there are ConfigServer and DataServer. ConfigServer can be detected through heartbeat. ConfigServer also has a host/standby relationship.

Several storage engines:

• MDB, full memory, can be used to store sessions and other data.
• RDB (similar to Redis), lightweight, eliminates operations like AOF and supports RESTfull operations
• LDB (LevelDB Storage Engine), persistent storage, LDB as the persistence of RDB, Google implementation, more efficient, theoretical basis is LSM(Log-Structured-Merge Tree) algorithm, now modify data in memory, When a certain amount of old data is written to disk together with the memory summary, it is written to disk again, which is more efficient for storage, compared to the Hash algorithm.
• Tair uses shared memory to store data, and if the service fails (not the server), the data will still be there after the service is restarted.

The message queue

• Message Queuing – Push/Pull Pattern Learning & ActiveMQ and JMS Learning

  • RabbitMQ consumers default to push mode (pull mode is also supported).
  • Kafka is in pull mode by default.
  • Push: The advantage is that the message can be sent to the consumer as quickly as possible, but the disadvantage is that if the consumer can’t keep up, the consumer’s buffer may overflow.
  • Pull mode: The advantage is that the consumer can be pulled according to the processing capacity, the disadvantage is that it will increase the message latency.
• “Comparison of Kafka, RabbitMQ, RocketMQ and Other Messaging Middleware — Message Delivery Performance and Difference”

The message bus

Message bus is equivalent to a layer of encapsulation on message queue, unified entry, unified control and simplified access cost.

• Message Bus vs. Message Queuing

Order of messages

• How to ensure the order in which consumers receive messages

RabbitMQ

Transactional support, push and pull modes are both supported and suitable for scenarios where reliable message transmission is required.

• Introduction to Application Scenarios and Basic Principles of RabbitMQ
• RabbitMQ for Message Queuing
• RabbitMQ Message Confirmation Mechanism (Transaction +Confirm)

RocketMQ

Java implementation, push and pull mode are supported, throughput is inferior to Kafka. Message order can be guaranteed.

• RocketMQ in Action Quick Start
• RocketMQ source code parsing

ActiveMQ

Pure Java implementation, JMS compatible, can be embedded in Java applications.

• Introduction to ActiveMQ Message Queuing

Kafka

High throughput, pull mode. Suitable for high IO scenarios, such as log synchronization.

• The official website
• “Comparison of message queues, Kafka in-depth analysis, recommended by all, wonderful article!”
• Introduction and Examples of Kafka Partitioning Mechanism

Redis newsfeed

Producer and consumer modes are completely client-side behaviors. List and pull modes are implemented, and BLPOP instructions are adopted to block waiting.

• Redis Learning Notes # 10: Redis as Message Queues

ZeroMQ

TODO

Timing schedule

Single machine timing scheduling

• Linux Timing Task Cron Configuration

• How Linux Cron Works

  • Fork process + sleep polling
• Summary of Quartz Usage
• Quartz source code analysis —- trigger start on time principle

• Quartz Principle Uncovering and Source Code Interpretation

  • In the QuartzSchedulerThread code, while() loops indefinitely. Each loop pulls out the next trigger and triggers the corresponding job until the scheduler thread is closed.

Distributed Scheduling

• These excellent domestic distributed task scheduling systems, how many have you used?

  • OpenCron, LTS, XXL-Job, Elastic-Job, Uncode-Schedule, Antares

• Fundamental Implementing Principles of Quartz Task Scheduling

  • In the Quartz cluster, individual Quartz nodes do not communicate with other Quartz nodes or management nodes, but are aware of the application of another Quartz through the same database table
• Elastic-Job-Lite
• Elastic-Job-Cloud

RPC

• Implementing the RPC Framework from scratch – RPC Principle and Implementation

  • Core roles: Server: the service provider that exposes the service; Client: the service consumer that calls the remote service; Registry: the Registry for service registration and discovery.
• “Performance comparison of distributed RPC frameworks against Dubbo, Motan, RPCX, GRPC, and Thrift”

Dubbo

• The official website
• The implementation principle of DUBBO is briefly introduced

SPI

TODO

Thrift

• The official website

• Thrift RPC Trail

  • Supports multiple languages and defines interfaces through intermediate languages.

gRPC

The server can authenticate and encrypt, and the data security can be guaranteed in the external network environment.

• The official website
• What You Should Know About RPC

Database middleware

Sharding Jdbc

• website
• The source code parsing

Logging system

Log collection

• Build an ELKB log collection system from scratch
• Build a Simple Log Collection and Analysis System with ELK
• “Log Collection System – Exploration”

Configuration center

• APOLLO – Ctrip open source configuration center application

  • Spring Boot and Spring Cloud
  • Support push and pull mode to update configuration
  • Support for multiple languages
• Unified configuration management based on ZooKeeper
• Spring Cloud Config Distributed Configuration Center Use Tutorial

The Servlet 3.0 asynchronous feature can be used to configure the client of the Center

• “New Servlet 3.0 Features — Asynchronous Processing”

API gateway

Main responsibilities: request forwarding, security authentication, protocol conversion, disaster recovery.

• “The API Gateway”
• On the Background, Architecture and Landing Scheme of API Gateway
• Building the API Gateway with Zuul
• Spring Cloud Gateway Source Code
• Introduction to Kong, one of the HTTP API Gateway Options

network

agreement

OSI seven layer protocol

• “OSI seven-layer protocol model, TCP/IP four-layer model learning notes”

TCP/IP

• TCP/IP Protocol
• Three Handshakes and Four Waves in TCP

HTTP

• “HTTP Protocol Detail (Hyperdetail)”

HTTP2.0

• “Detailed Analysis of HTTP 2.0 Principles”

• “The basic unit of HTTP2.0 is binary frames”

  • Use binary frames for transmission.
  • Multiplexing.

HTTPS

• Popular Understanding of the Principles of HTTPS

  • Asymmetric encryption negotiation encryption algorithm is used
  • Use symmetric encryption to transfer data
  • Use a certificate issued by a third party organization to encrypt the public key for secure transmission of the public key and prevent it from being changed by a middleman.
• Eight Free SSL Certificates – Add HTTPS Secure Encryption to Your Website for Free

A network model

• Five models of I/ O and three modes of working on the Web: Principles you must understand for Web optimization

  • Five types of I/O: blocking I/O, non-blocking I/O, I/O multiplexing, event-driven I/O, and asynchronous I/O. The first four types of I/O are synchronous operations, the first phase of I/O is different, the second phase is the same, and the last type is asynchronous.
  • There are three working modes of Web Server: Prefork(multi-process), Worker (thread) and Event.

• A summary of the differences between select, poll and epoll

  • Select, poll, and epoll are essentially synchronous I/O in that they are responsible for reading and writing after the read and write event is ready, which means that the read and write process is blocked.
  • The default number of open file descriptors is 1024 (2048 for x64). Polls use a linked list structure and are unlimited in number.
  • Select, poll “awake” time to traverse the entire collection of fd, and epoll when “awake” as long as the judge whether the ready list is empty, save a lot of CPU time by callback mechanism; Each call to select, poll copies the fd set from user to kernel state once, whereas epoll only copies it once.
  • Poll will gradually degrade with the increase of concurrency, while epoll adopts a red-black tree structure, which ensures stable performance and will not decrease with the increase of connections.

• Select, poll, epoll compare

  • In cases where the number of connections is small and both connections are active, select and poll may perform better than epoll, whose notification mechanism requires a lot of function callbacks.

• “Deep Understanding of Java NIO”

  • NIO is a synchronous non-blocking IO model. Synchronized means that the thread is constantly polling for IO events to be ready. Non-blocking means that the thread can do other tasks while waiting for the IO
• Differences between BIO, NIO and AIO
• Two Efficient Server Design Models: Reactor and Proactor

Epoll

• “Epoll Usage Details (Essence)”

Java NIO

• “Deep Understanding of Java NIO”
• “An Example of Java NIO Writing Socket Server”

kqueue

• Introduction to kqueue usage

Connections and short connections

• TCP/IP series – the difference between long and short connections

The framework

• Anatomy of Netty Principles

  • Reactor model introduction.
  • Netty is an implementation of the Reactor pattern.

Zero copy

• Understanding Zero Copy of Netty Bytebuf

  • Multiple physically separate buffers are logically merged into one, thus avoiding the need to copy data from memory to memory.

Serialization (binary protocol)

Hessian

• Binary-RPC; Binary-RPC; It’s not just serialization

Protobuf

• ProtoBuf is an example of the protocol in Java. It takes up more space and is more efficient than other serialization libraries such as Hessian. You need to write the.proto file.

• Protocol Buffers Serialization Protocol and Applications

  * Interpretation of the Agreement; Disadvantages: poor readability;

• Simple Use of ProtoBuf and ProtoStuff

  • The nice thing about ProtoStuff is that Java objects can be serialized without having to write.proto files.

The database

The basic theory

Three paradigms for relational database design

• Three Paradigms and Five Constraints for Databases

  • The first normal form: each column (each field) in the data table must be the smallest unit that cannot be split, that is, to ensure the atomicity of each column;
  • Second normal form (2NF) : After 1NF is satisfied, all columns in the table must be dependent on the primary key, and no column can be unrelated to the primary key, that is to say, a table only describes one thing;
  • Third normal form: The second normal form (2NF) must be satisfied first, which requires that each column in the table is only directly related to the primary key, not indirectly, (each column in the table can only depend on the primary key);

MySQL

The principle of

• InnoDB indexing principle for MySQL

• MySQL Storage Engine –MyISAM vs. InnoDB

  • The main difference between the two types is that InnoDB supports transactions with foreign keys and row-level locking
• MyISAM and InnoDB Index Implementation Differences

InnoDB

• Understanding MySQL and InnoDB in an article

To optimize the

• MySQL36 Catch-22
• The best 20+ MySQL performance optimization lessons
• “The Path of SQL Optimization”
• Causes and Solutions of MySQL Database Deadlocks
• Possible circumstances that cause index failure

• MySQL paging limit speed optimization method

  • The principle is to narrow down the scanning area.

The index

Clustered index, non-clustered index

• MySQL clustered index/non-clustered index
• Indexing Implementation for MyISAM and InnoDB

MyISAM is non-aggregated, InnoDB is aggregated

The composite index

• Advantages and Precautions of Composite Indexes

  • There is an error in the text:

    For the compound index, it is best to use the condition order in the order of finding the index, which is the most efficient. Select * from table1 where col1=A AND col2=B AND col3=D select * from table1 where col2=B AND col3=D

  • SELECT * FROM ‘col1’ WHERE ‘col2’ = ‘B’ AND ‘col1’ = ‘A’; SELECT * FROM ‘col1’ WHERE ‘B’ = ‘B’ AND ‘col1’ = ‘A’; Col1 =A AND col2=B, so the index will still be used.
• MySQL > SELECT * FROM ‘WHERE’ WHERE ‘FROM’ WHERE ‘WHERE’ FROM ‘WHERE’;

Adaptive Hash Index (AHI)

• InnoDB Storage Engine — Adaptive Hash Indexing

explain

• “MySQL Performance Optimizer Explain Usage Analysis”

NoSQL

MongoDB

• Directing a tutorial

• The pros and cons of MongoDB versus relational databases

  • Advantages: weak consistency (final consistency), better guarantee the user access speed; Built-in GridFS, support large storage capacity; Schema-less database, no predefined structure; Built-in Sharding. Compared to other NoSQL, third-party support is rich; Superior performance;
  • Disadvantages: MongoDB does not support transactional operations; MongoDB occupies too much space; MongoDB does not have as mature maintenance tools as MySQL, which is a noticeable point for both development and IT operations;

Hbase

• “A Brief Introduction to HBase”
• “In-depth Learning of HBase Architecture Principles”
• “The difference between traditional row storage and (HBase) column storage”

• “Differences between HBase and Traditional Database”

  • Empty data is not stored, saves space, and is suitable for concurrency.

• HBase Rowkey Design

  • Rowkeys are arranged in lexicographical order to facilitate bulk scanning.
  • Hotspots can be avoided by hashing.

Search engine

Principles of search engines

• Inverted Indexing – A Search Engine Introduction

Lucene

• Introduction to Lucene

Elasticsearch

• Elasticsearch: Elasticsearch: Elasticsearch:
• Elasticsearch indexing principle

Solr

• Apache Solr Getting Started
• Elasticsearch vs. Solr

sphinx

• Introduction to Sphinx and Explore Its Principles

performance

Performance optimization methodology

• “15 days of performance tuning, 5 aspects of tuning experience”

  • Code level, business level, database level, server level, front-end optimization.
• Several Aspects of System Performance Optimization

Capacity assessment

• Methodology and Typical Cases of Networking Performance and Capacity Evaluation

• “Internet Architecture, How to Design Capacity?”

  • Evaluate the total traffic, evaluate the average traffic QPS, evaluate the peak QPS, evaluate the system, and evaluate the single machine limit QPS

CDN network

• Principle of CDN Acceleration
• What are the best CDN in China?

The connection pool

• Comparison and development configuration of mainstream Java database connection pool

Performance tuning

• 9 Java Performance Debugging Tools, Must Have At Least One

Big data

Flow calculation

Storm

• The official website
• The most detailed introduction to Storm

Flink

• Introduction to the Basic Principles of Flink

Kafka Stream

• Kafka Stream Research: A Lightweight Stream Computing Model

Application scenarios

Such as:

• Real-time statistics related to advertising;
• Real-time update of user portrait label in recommender system;
• Real-time health monitoring of online services;
• Real-time charts;
• Real-time data statistics.

Hadoop

• What Hadoop Is and Can Do in Plain English
• The most detailed Hadoop environment ever built

HDFS

• “Fundamentals of HDFS”

MapReduce

• “Map/Reduce Principles in Plain English”
• “Simple Map-Reduce Java Example”

Yarn

• Understanding the Architecture and Principles of YARN

Spark

• Spark Part 1: Basic Architecture and Principles
• Spark Introduction to Subrain Big Data (Python)

security

Web security

XSS

• “XSS attack principles and solutions”

  CSRF

• Principle and Prevention of CSRF

SQL injection

• SQL Injection

Hash Dos

• “Evil Java Hash DoS Attack”

  • Upload large JSON using JSONObject. The underlying JSONObject uses HashMap. Different data results in the same hash value, which slows building hashes and consumes CPU.
• An Advanced DoS Attack -Hash Collision Attack
• “On Hash Collision DOS Vulnerability: Analysis and Solution”

Script injection

• Principle and Prevention of File Upload Vulnerability

Vulnerability scanning tool

• “DVWA”
• W3af
• OpenVAS,

Verification code

• Principle Analysis and Implementation of Captcha

• “Detailed implementation principle of sliding verification code”

  • Sliding CAPTCHA evaluates risk based on the response time of the person sliding the slider, drag speed, time, position, trajectory, retry times, etc.
• Research on Taobao Sliding Verification Code

To prevent DDoS

• “Study Manual: DDoS Attack and Defense Means”
• Free DDoS Attack Testing Tools Collection

User privacy information protection

1. The user password is saved in plaintext, and a dynamic salt is added.
2. ID card number, mobile phone number if to display, with “*” to replace part of the character.
3. Whether the contact information is displayed or not is controlled by the user.
4. TODO

• What Personal Privacy Includes
• What is the scope of privacy on the Internet?
• “User password saving”

Serialization vulnerability

• Analysis of Common Utilization of Java Desialization Vulnerability in Lib

Encryption to decrypt

Symmetric encryption

• Common Symmetric Encryption Algorithms

  • DES, 3DES, Blowfish, AES
  • DES uses 56-bit secret keys, Blowfish uses variant-length keys from 1 to 448 bits, and AES 128,192 and 256 bits.
  • DES secret key is too short (only 56 bits) algorithm has been replaced by AES, and AES has hardware acceleration, performance is very good.

The hash algorithm

• “Common Hashing Algorithms”

  • MD5 and SHA-1 are no longer secure and have been deprecated.
  • Currently SHA-256 is relatively secure.
• Design of Public Network URL Signature Verification Based on Hash Summary Signature

Asymmetric encryption

• Common Asymmetric Encryption Algorithms

  • RSA, DSA, ECDSA(helical curve encryption algorithm)
  • Different from RSA, DSA can only be used for digital signature, and cannot encrypt and decrypt data. Its security is comparable to RSA, but its performance is faster than RSA.

  • A 256-bit ECC secret key is as secure as a 3072-bit RSA secret key.

    “Blockchain Encryption Technology”

Server security

• Linux Enhancement: 15 Steps to Building a Secure Linux Server

Data security

The data backup

TODO

Network isolation

Internal and external network separation

TODO

Log on to the jumper

Log on to the online host via the jumper in the internal and external environment.

• Building a Simple Fortress Machine

Authorization, authentication

• Authorization authentication knowledge base

RBAC

• Permission design based on organizational roles
• Permission System and RBAC Model Overview
• “A detailed case study of Spring’s integration with Shiro to make permission control module”

OAuth2.0

• Understanding OAuth 2.0
• OAuth2.0 in One Picture

OIDC

• Understand OIDC

SAML

• Understand the SAML

Two-factor certification (2FA)

2FA-Two-Factor Authentication, used to enhance login authentication

The common practice is to use the login password + mobile phone verification code (or token Key, similar to the USB Key for online banking).

• [book a course of two-factor authentication (fa) 2] (http://www.ruanyifeng.com/blo…

Single Sign-On (SSO)

• Principle and Simple Implementation of Single Sign-On
• CAS single sign-on framework
• Use Authing to implement single sign-on

Common open source frameworks

Open source licenses

• Open Source Protocol Choice
• How do I choose an open source software protocol

The logging framework

Log4j, Log4j2

• “Log4J in detail”
• “Log4j2 in action detail”

• “Log4J1,Logback, and Log4J2 Performance Testing Comparison”

  • Log4J asynchronous logging has excellent performance.

Logback

• The most complete LogBack explanation, including Java examples and configuration instructions

ORM

• The pros and cons of using an ORM framework

  • The main purpose is to improve development efficiency.

MyBatis:

• “MyBatis cache mechanism detail”

  • The Level 1 cache is the SQLSession level cache, and the cached data is only valid within the SQLSession
  • The second level cache is a mapper level cache that is common to the same namespace, so it is shared for SQLSessions. Use the LRU mechanism to clean up the cache and turn it on with the cacheEnabled parameter.
• Code Generator for MyBatis Learning

Network framework

TODO

Web framework

The family of the Spring

Spring

• Spring Tutorial

Spring Boot

• The official website
• Spring Boot Basics Tutorial

Spring Cloud

• Spring Boot Chinese index station
• Spring Cloud Chinese document
• Spring Cloud Basics Tutorial

Tool framework

• Introduction and Simple Use of Apache Commons Tool Classes
• Google Guava Chinese Tutorial

Distributed design

Extensibility design

• Top 10 Extensible Architectures an Architect Can’t Know

  • In summary, the common approach is distribution, caching, and asynchronous processing.

• Data Sharding for Scalable Design

  • Horizontal segmentation + vertical segmentation
  • Use middleware for sharding, such as MySQL Proxy.
  • Sharding using a sharding strategy, such as modulo by ID.

• “How to build scalable large websites”

  • Distributed services + message queues.
• “Technical Architecture of Large Websites (VII) – Site Extensibility Architecture”

Stability & high availability

• System Design: Some Technical Solutions for High Availability Systems

  • Scalable: Horizontal scale, vertical scale. Avoid a single point of failure through redundant deployment.
  • Isolation: Avoid a single business occupying all resources. 2. Isolation of machine room to avoid single point of failure.
  • Decoupling: Reduces maintenance costs and reduces coupling risks. Less dependence, less mutual influence.
  • Current limiting: sliding window counting method, leaky bucket algorithm, token bucket algorithm and other algorithms. In case of sudden flow, ensure the stability of the system.
  • Downgrade: Releasing resources that are not core functions in an emergency. Sacrifice non-core business to ensure high availability of core business.
  • Fuse: the abnormal situation exceeds the threshold value and enters the fuse state and fails quickly. Reduce the impact of unstable external dependencies on core services.
  • Automated testing: Reduce release-induced failures through sound testing.
  • Grayscale publishing: Grayscale publishing is a compromise between speed and security, which can effectively reduce publishing failures.

• About High Availability Systems

  • Design principle: data not lost (persistence); Service high availability (service copy); Absolute 100% high availability is difficult, the goal is to achieve as many 9’s as possible, such as 99.999% (a total of only 5 minutes for the whole year).

Hardware load balancing

• Comparison of Advantages and Disadvantages of NGINX and F5 Load Balancer Technologies

  • Mainly compared to F5.
• What Do You Know About Software/Hardware Load Balancing Products?

Software load balancing

• “Several load balancing algorithms” round search, weight, load, minimum connection, QoS

• DNS Load Balancing

  • Simple configuration, slow update speed.

• Nginx Load Balancing

  • Simple and light weight, low learning cost; Mainly suitable for web applications.

• Load Balancing with LVS+Keepalived

  • Configure the comparison load, only support to 4 layers, high performance.

• HAProxy Usage Details The most detailed Chinese document on the web

  • Support to seven layers (such as HTTP), more comprehensive features, performance is good.

• HAProxy +Keepalived+MySQL Load Balancing

  • It is mainly load balancing of user read requests.
• RabbitMQ + HAProxy + Keepalived to build a highly available cluster

Current limiting

• “On Current Limiting in Highly Concurrent Systems”

  • Counter: by sliding window counter, control the number of requests per unit time, simple and rough.
  • Leakage bucket algorithm: a leaky bucket with a fixed capacity. When the leaky bucket is full, the request is discarded. It is commonly used.
  • Token bucket algorithm: Token bucket with a fixed capacity is added at a certain rate. Tokens need to be obtained before processing the request. If the request cannot be obtained, the request will be discarded or entered into the loss queue. The rateLimiter in Guava is an implementation of a token bucket.
  • Nginx Current Limiting: Passlimit_reqAnd other modules to limit the number of concurrent connections.

Application layer disaster tolerance

• Avalanche Weapon: The Principle and Use of the Fuse Hystrix

  • Avalanche effect causes: hardware failures, hardware failures, program bugs, retries increase traffic, user requests.
  • Avalanche countermeasures: limit the flow, improve the cache mode (cache preload, synchronous call to asynchronous), automatic capacity expansion, downgrade.

  • Hystrix Design Principles:

    • Resource isolation: Hystrix avoids a service avalanche by allocating each dependent service to a separate thread pool for resource isolation.
    • Fuse switch: Health of the service = number of requests failed/total number of requests, controlled by threshold setting and sliding window.
    • Command pattern: Wrapping the service invocation logic by inheriting HystrixCommand.
• Cache Penetration, Cache Breakdown, Cache Avalanche Solution Analysis

• Cache breakdown, invalidation, and hot key issues

  • Main strategies: failure moment: single machine use lock; Use distributed locks; Not expired;
  • Hotspot data: Hotspot data is stored separately; Use local caches; Divide into multiple sub-keys;

Disaster tolerance across the machine room

• Experience on the Deployment of “Multi-living in Different Areas” Multi-room

  • Data synchronization is carried out through self-developed middleware.

• Practical Experience on Multi-activity in Different Regions (Double Activity in Different Regions)

  • Be aware of delays, as multiple calls across the machine room will magnify the delay several times.
  • Building room special line is very likely to have problems, do a good job of operation and procedure level of fault tolerance.
  • Can not rely on the program side data double write, to have automatic synchronization scheme.
  • Data will never be synchronized with high latency and poor network quality.
  • The core business and the secondary business are divided and subjugated, or even only the core business is considered.
  • Remote multi-active monitoring deployment, testing should also keep up.
  • Consider user partitions where business permits, especially games and email services.
  • Control the message body size across the machine room, the smaller the better.
  • Consider using Docker container virtualization technology to improve dynamic scheduling capabilities.
• Introduction of disaster recovery technology and construction experience

Disaster recovery drill process

• “Dependency management, grayscale release, fault drill, Ali e-commerce fault drill system design and practical experience”

  • Common Fault Portrait
  • Cases: plan validity, plan validity, failure recurrence, architecture disaster tolerance test, parameter tuning, parameter tuning, fault surprise, joint drill.

A smooth start

• 2. Flush data (if any). 3. Restart the app
• System. Exit, Kill SIGTERM. How to shut down Java services gracefully. Kill -9 is not recommended; Register the hook with Runtime.addShutdownHook.
• “How to close common Java applications gracefully” Java, Spring, Dubbo elegant shutdown methods.

Database expansion

Read-write separation mode

• Implementation of MySQL Master and Slave Scheme
• Build MySQL Master/Slave Copy Classic Architecture
• “HAProxy + Load Balancing with Multiple MySQL Slave”

• DRBD+Heartbeat+ MySQL High Availability Read/Write Detach Architecture

  • DRDB does disk replication and avoids a single point of problem.
• MySQL Cluster: MySQL Cluster

Fragmentation patterns

• Problems and Schemes to be Considered in the Classification of Database and Table

  • Middleware: Lightweight: Sharding-JDBC, TSharding; Heavyweights: Atlas, MyCat, Vitess, etc.
  • Issues: Transactions, Join, Migration, Scaling, ID, Paging, etc.
  • Transaction compensation: check the reconciliation of data; Log based comparison; Synchronize with standard data sources on a regular basis.
  • Sublibrary strategy: numerical range; Modulus; The date, etc.
  • The number of branches: Usually 50 million for a single MySQL database and 100 million for a single Oracle database need branches.

• MySql > table partitioning

  • Partitioning: An internal MySQL mechanism that is transparent to the client and allows data to be stored in different files, ostensibly in the same table.
  • Separate tables: Different tables are physically created and clients need to manage the routing of separate tables.

Service governance

Service registration and discovery

• Never Lost! How to Implement Service Discovery in Microservice Architecture?

  • Client service discovery mode: The client queries the registry directly and is responsible for load balancing itself. Eureka takes this approach.
  • Server-side service discovery pattern: Clients query service instances through load balancing.

• SpringCloud service registry comparison :Consul vs ZooKeeper vs ETCD vs Eureka

  • CAP Support: Consul (CA), ZooKeeper (CP), ETCD (CP), Euerka (AP)
  • The author thinks that Consul currently has better support for Spring Cloud.

• Service Registration and Discovery Based on ZooKeeper

  • Advantages: simple API, Pinterest, Airbnb in use, multi-language, through the Watcher mechanism to achieve configuration PUSH, can quickly respond to configuration changes.

Service routing control

• Distributed Service Framework Learning Note 4 Service Routing

  • Principle: Transparent routing
  • Load balancing strategies: randomness, polling, service invocation latency, consistent hashing, sticky connections
  • Local routing priority policy: injvm(call services within the JVM first), innative(use services on the same physical machine first), in principle, find the nearest service.
  • Configuration mode: unified registry; Local configuration; Issue dynamically.

Distributed consistency

Cap and BASE theory

• From Distributed Consistency to CAP Theory and Base Theory

  • Consistency classification: strongly consistent (immediately consistent); Weak consistency (consistency can be achieved in unit time, such as seconds); Final agreement (a form of weak agreement in which final agreement occurs over a period of time)
  • CAP: Consistency, Availability, Partition Fault Tolerance (Caused by Network Failure)
  • Base: Basically Available, Soft State, and Eventually Consistent
  • The core idea of BASE theory is: even if strong consistency cannot be achieved, each application can adopt appropriate methods to achieve final consistency according to its own business characteristics.

A distributed lock

• Several Implementations of Distributed Lock

  • Database-based distributed locking: advantages: simple operation, easy to understand. Disadvantages: single point problem, database performance enough overhead, non-reentrant;
  • Buffer-based distributed locking: Advantages: non-blocking, good performance. Disadvantages: the operation is not easy to cause the lock can not be released.
  • ZooKeeper Distributed Lock: The locking mechanism is implemented through ordered temporary nodes. If the corresponding node needs the minimum, the lock is considered to have been obtained. Advantages: Clustering can solve the single point problem transparently, avoid the problem of lock not being released, and the lock can be reentrant. Disadvantages: Performance is not as good as caching, throughput decreases as the size of the ZK cluster grows.

• Distributed Lock Based on ZooKeeper

  • Clear principle description + Java code examples.

• Jedislock — Redis Distributed Lock Implementation

  • Setnx (set if ont exists) returns false, otherwise returns true. Expiration time is supported.

• Memcached and Redis Distributed Lock Solution

  • Using memcached’s add (as opposed to set) operation returns false when the key is present.

Distributed consistency algorithm

PAXOS

• Distributed Series – Principles and Derivations of Paxos Algorithms
• “Paxos–>Fast Paxos–> ZooKeeper Analysis”
• [Distributed] ZooKeeper and Paxos

Zab

• Zab: An Introduction to Distributed Consistency Protocol in ZooKeeper

Raft

• Why RAFT Is an Easier to Understand Distributed Consistency Algorithm

  • Three types of roles: Leader, Follower and Candidate
  • Votes are sent out by random waiting, and the one with the most votes wins.

Gossip

• Gossip Algorithm

Two-phase commit, multi-phase commit

• “About Distributed Transactions, Two-Phase Commit Protocol, Three-Stage Commit Protocol”

Power etc.

• “Distributed Systems — Idempotency Design”

  • What the idempotent feature does: The resource is idempotent, and requestors do not have to worry about repeated invocations causing errors.
  • Common means of guaranteeing idempotency: MVCC (similar to optimistic locking), deduplication table (unique index), pessimistic locking, one-time token, serial number method.

Distributed consistent scheme

• Transaction Consistency Solution for Distributed Systems
• Six Schemes to Ensure Data Consistency in Distributed Systems

Distributed Leader node election

• “Implementation of Distributed Leader Node Election with ZooKeeper”

TCC(Try/Confirm/Cancel) flexible transactions

• Traditional and Flexible Transactions

  • Base theory: basic availability, flexible state, final consistency.
  • Solution: log + compensation (forward replenishment or rollback), message retry (require the program to be idempotent); “Lock free design”, using optimistic locking mechanism.

Distributed file system

• Distributed file storage system – basic architecture?

• Comparison of Distributed File Systems?

  • HDFS: Large-volume data reading and writing, for high-throughput scenarios, not suitable for small files.
  • FastDFS: Lightweight and suitable for small files.

Unique ID generation

Globally unique ID

• Generating global unique ID summaries in highly concurrency distributed systems

  • Twitter scheme (Snowflake algorithm) : 41-bit timestamp + 10-bit machine identity (such as IP, server name, etc.) + 12-bit serial number (local counter)
  • MySQL > REPLACE INTO XXX:SELECT LAST_INSERT_ID();
  • UUID: Shortcoming, unordered, string is too long, takes up space, affects retrieval performance.
  • MongoDB scenario: Leverage ObjectId. Disadvantages: not self-increasing.

• Principle of Sequence for TDDL in Distributed System

  • Create a SEQUENCE table in the database to record the maximum number of currently occupied IDs.
  • Each client host takes an ID interval (for example, 1000~2000) and cages it locally, and updates the ID maximum record in the SEQUENCE table.
  • Different ID intervals are taken between the hosts of the client, and then taken after they are used. Optimistic locking mechanism is used to control concurrency.

Consistent Hash algorithm

• Consistent Hashing Algorithm

Design Idea & Development Pattern

DDD(Domain-Driven Design – Domain Driven Design)

• A Brief Discussion on My Understanding of DDD Domain Driven Design

  • Concept: DDD is mainly put forward to solve the problem of fragmentation in the various stages of the traditional software development process (analysis – design – code), so as to avoid the problem that software cannot be delivered (and the demand side’s assumption is inconsistent) due to unclear analysis at the beginning or inconsistent information flow during the software development process. DDD emphasizes that everything is Domain-centric, and the role of Domain Expert is emphasized. It emphasizes that the development takes place after the Domain model is defined first, and that the Domain model can guide the development (the so-called driver).
  • Process: Understand the domain, split the domain, refine the domain, the accuracy of the model depends on the depth of understanding of the model.
  • Design: Modeling tools such as aggregations, entities, value objects, factories, warehouses, domain services, and domain events are proposed in DDD to help with domain modeling.

• Summary of Fundamentals of Domain Driven Design

  • Domain (Doamin) is essentially a problem domain, such as an e-commerce system, a forum system, etc.
  • To give a Bounded Context a representation of the relationships between subdomains, which can be understood simply as a subsystem or component module.
  • Domain Model: The core of DDD is to establish the correct Domain Model (using common description language, tool-domain common language); Reflect the nature of business requirements, including entities and processes; It runs through the whole process of software analysis, design and development. Common ways to express domain models: diagrams, code, or words;
  • Domain Common Language: A language or tool that can be understood by domain experts, developers and designers.
  • Classic layered architecture: user interface/presentation layer, application layer, domain layer, infrastructure layer, is a four-tier architecture pattern.

  • Patterns used:

    • As few correlations as possible, as single item as possible, as low as possible overall complexity.
    • Entity: A unique identifier in a domain. An Entity has as few attributes as possible, and as few as possible are clear.
    • Value Object: Small, simple objects with no unique identity and immutable property values, such as Date.
    • Domain Service: Coordinates multiple Domain objects, only methods have no state (no data); It can be divided into application layer services, domain layer services, basic layer services.
    • Aggregate and Aggregate Root (Aggregate, Aggregate Root) : An Aggregate defines a set of related objects that have a cohesive relationship; The aggregation root is the only element referenced to the aggregation; When modifying an aggregation, it must be at the transaction level; In most domain models, 70% of aggregations typically have only one entity and 30% have 2-3 entities; If an aggregation has only one entity, then that entity is the aggregation root. If there are multiple entities, then we can think about which object in the aggregate has a sense of independent existence and can directly interact with the outside;
    • Factory: Similar to the Factory pattern in design patterns.
    • Repository: Permits to DB, manages objects, and designs repositories for aggregation only.

• The Path to Domain Driven Design (DDD) Implementation

  • Aggregation: For example, a Car contains components such as Engine, Wheel and Tank. None can be dispensable.
• Domain Driven Design Series (2) Brief Analysis of the Concept, Difference and Uses of VO, DTO, DO and PO

Command Query Separation of Responsibilities (CQRS)

CQRS — Command Query Responsibility Seperation

• Domain Driven Design Series 6: CQRS

  • Core Idea: Read/write separation (query and update in different methods), different processes are just different designs, separation of CQ code is evident in distributed environments (where there is redundant data), for high performance.

• Comparison of Advantages and Disadvantages between DDD CQRS Architecture and Traditional Architecture

  • The final consistent design concept; Rely on highly available messaging middleware.

• Introduction to CQRS Architecture

  • An abstract example of implementing CQRS.

• “My Thoughts on the CQRS/EventSourcing Architecture”

  • CQRS model analysis + 12306 ticket snatching case

Anemia, congestion model

• “Anemia and congestion model interpretation and some experience”

  • Blood loss model: Laozi and son are defined separately, but do not know each other. There is no business logic in the entity definition of the two, and they are related through external Service.
  • Anemia model: Lao Tzu knows son, son also knows Lao Tzu; Part of the business logic is placed in entities. Advantages: Each layer single dependent, clear structure, easy to maintain; Disadvantages: Doesn’t conform to OO mentality, the Service layer is heavier than congestion mode;
  • Congestion model: Similar to the anemia model, the difference is how the business logic is divided. Advantages: The Service layer is thin, acting as a Facade, not dealing with DAOs, compound OO mentality; Disadvantages: Non-monolithic dependencies, bidirectional dependencies between DO and DAO, and logical division of the Service layer can cause confusion.
  • Swelling mode: An extreme case where the Service layer is removed and all business logic is placed in DO. Advantages: in line with OO thought, simplified stratification; Cons: Too much information is exposed, and a lot of non-DO logic can be forced into DO. This pattern should be avoided.
  • The authors advocate the use of anemic models.

The Actor model

TODO

Reactive programming

Reactor

TODO

RxJava

TODO

Vert.x

TODO

DODAF2.0

• DoDAF 2.0 Methodology
• DoDAF 2.0: How the Power Perspective Landing

Serverless

There is no need for too much of a service architecture concept of a relational server.

• What is Serverless Serverless Architecture?

  • Serverless does not mean going out to the server, but removing the concern about the health of the server.
  • Serverless represents a shift in thinking from “building a set of services on a server to respond to multiple events to building a server to respond to one event”.
  • Serverless does not represent a specific framework.

• How to Understand Serverless?

  • Depends on BAAS ((Mobile) Backend as a Service) and FAAS (Functions as a Service)

Service Mesh

• What is a Service Mesh?
• Introduction to the Service Mesh

The project management

Architecture review

• Instructions on how to review architecture design for architecture design
• Everyone is an Architect: Non-functional Requirements

refactoring

• 12 Catch-22 Rules for Restructuring Architecture

Code specification

• Alibaba Java Development Manual

Code Review

The system is the system! In addition, each company needs to develop its own check list according to its own needs and goals

• Why Are You Bad at Code Review?

  • Code Review does a good job in institutional building.
• Code Review from Zero
• Code Review Checklist
• Java Code Review Checklist
• How to Do Code Review with GitLab

RUP

• “Software Architecture Design Using RUP 4+1 View Method”

Kanban management

• “Talk about Kanban in Projects”

SCRUM

The SCRUM – jump ball

• 3 roles :Product Owner(PO) Product Owner; Scrum Master (SM) to drive Scrum execution; Team development Team.
• Three artifacts: Product Backlog Product Todolist, including priority; Sprint Backlog feature development Todo List Burnout figure;
• Five values: Focus, Courage, Openness, Commitment and Respect.
• Agile Project Management Processes – A Complete Summary of the Scrum Framework!
• Agile is Simple 3- Scrum for Agile Methods

Agile development

TODO

Extreme Programming (XP)

XP – eXtreme Programming

• Mainstream Agile: Extreme Programming XP

  • It is a methodology for guiding developers.

  • Four values:

    • Communication: Encourage verbal communication to improve efficiency.
    • Simple: Just enough.
    • Feedback: Feedback and inform relevant people in time.
    • Courage: Embrace change and dare to reinvent.
  • The five principles: quick feedback, simple assumptions, progressive change, advocate change (small steps), and good work (small steps without quality).
  • Five tasks: Stage Sprint; Sprint planning meetings; Daily stand-up meetings; Post-sprint review; Review the meeting.

Pair programming

While writing code, review. Enhance code quality and reduce bugs.

• Pair Programming

PDCA cycle quality management

A — PLAN B — DO C — CHECK D — ACT

• “PDCA”

FMEA management mode

TODO

General business terms

TODO

Technology trends

TODO

Policies and regulations

The law

• Cyber Security Law of the People’s Republic of China

  • Issued on November 7, 2016 and effective as of June 1, 2017

• Personal Information Protection Act

  • The Personal Information Protection Law, a legal provision to protect personal information, is still being formulated. In the information security-related proposals for the 2019 NPC and CPPCC sessions, CPPCC members have called for attention to privacy protection in the era of big data and accelerated legislation.

• Interpretations of the Supreme People’s Court and the Supreme People’s Procuratorate on Several Issues Concerning the Application of the Law in Handling Criminal Cases of Infringement of Citizens’ Personal Information

  • The “Interpretation” consists of 13 articles and will come into force on June 1, 2017

    • 1, for the track information, communication content, credit information, property information, illegal access to, sell or provide more than 50 pieces of information is considered “serious”;
    • 2. For accommodation information, communication records, health and physiological information, transaction information and other personal information that may affect the safety of personal and property, the standard is more than 500 pieces;
    • 3. For other citizens’ personal information, the standard is more than 5000 pieces.

• Electronic Commerce Law of the People’s Republic of China

  • On August 31, 2018, the fifth session of the Standing Committee of the 13th National People’s Congress (NPC) voted to adopt the e-commerce law, which will take effect on January 1, 2019
  • Interpretation of E-commerce Law (1) What is E-commerce
  • Interpretation of e-commerce law (2) e-commerce operators
  • Interpretation of E-commerce Law (III) Code of Conduct of E-commerce
  • Interpretation of e-commerce law (IV) the legal relationship of e-commerce
  • Interpretation of e-commerce law (outside) electricity business to make money secret
  • Interpretation of E-commerce Law (Outsider) E-commerce Model
• Common sense of law that programmers need to know
• Vernacular Law 42 – an exclusive legal weapon for programmers

Strictly abide by Article 253 of the Criminal Law

Article 253 of the Criminal Law of China stipulates:

• State organs or financial, telecommunications, transportation, education, medical and other units of staff, in violation of the provisions of the state, will this unit in the process of perform their duties, or providing services for citizens personal information, sell or illegally provide to others, if the circumstances are serious, shall be sentenced to fixed-term imprisonment of not more than three years or criminal detention, and concurrently or independently be sentenced to a fine.
• Whoever steals or illegally obtains the above-mentioned information by other means, if the circumstances are serious, shall be punished in accordance with the provisions of the preceding paragraph.
• Where a unit commits any of the crimes mentioned in the preceding two paragraphs, it shall be fined, and the persons who are directly in charge and the other persons who are directly responsible for the crime shall be punished in accordance with the provisions of the respective paragraphs.

The Supplementary Provisions of the Supreme People’s Court and the Supreme People’s Procuratorate on the Enforcement of the Crimes of the Criminal Law of the People’s Republic of China stipulate that: Violation of Article 253 (1) of the Criminal Law constitutes “the crime of selling or illegally providing citizens’ personal information”; Violated paragraph 2 of Article 253 of the Criminal Law, constitutes “the crime of illegally obtaining citizens’ personal information”.

• Crime of Unlawful Acquisition of Personal Information of Citizens

Harbor principle

The principle of “safe harbor” refers to the fact that in the event of copyright infringement, when an ISP (Internet Service Provider) only provides a space service and does not produce web content, the ISP is obliged to remove it if it is notified of the infringement, otherwise it is deemed to have infringed. The ISP is not liable for infringement if the infringing content is not stored on the ISP’s servers and is not informed of what should be removed. Later, the principle of safe haven has also been applied to search engines, network storage, online libraries and other aspects.

• The Principle of Haven

Quality of Architect

• Portrait of the Architect

  • Business understanding and abstraction
  • NB’s code capabilities
  • Comprehensive: 1. Whether multiple technical solutions emerge in the architect’s mind when facing business problems; 2. 2. Whether enough aspects have been taken into consideration when designing the system; 3. Whether enough aspects have been taken into consideration when designing the system;
  • Global: Are the impacts on upstream and downstream systems considered?
  • 3. To weigh the input-output ratio; Priority and pacing control;

• “What architects need to know about architecture optimization and design”

  • Details to consider: modular, lightly coupled, shared-nothing architecture; Reduce the dependencies of various components, pay attention to the chain of failures caused by dependencies between services, and the impact.
  • Comprehensive consideration of infrastructure, configuration, testing, development, operation and maintenance.
  • Consider the impact of people, teams, and organizations.
• “How can I really improve myself to be a good architect?”

• The essential qualities and ways to grow as an architect

  • Quality: business understanding, technical breadth, technical depth, rich experience, communication ability, practical ability, aesthetic quality.
  • Growth path: 2 years to accumulate knowledge, 4 years to accumulate skills and influence within the group, 7 years to accumulate influence within the department, and more than 7 years to accumulate influence across departments.

• The Architect — What Floor Are You On?

  • The first-level architect sees only the product itself
  • The second-tier architect sees not only his own product, but also the overall solution
  • The third-tier architect sees the business value

Team management

TODO

recruitment

information

Industry information

• 36kr
• Techweb

List of Public Accounts

TODO

blog

Team blog

• Alibaba Middleware Blog
• Meituan comments on the technical team blog

Personal blog

• Ruan Yifeng’s weblog
• Cool Shell – Coolshell – Chen Hao
• Hellojava – Ari Bishuan
• Cm’s Blog
• DD- Zhai Yongchao – author of “Spring Cloud Microservice Practice”

Integrated portal, community

Domestic:

• CSDN old technology community, no need to explain.
• 51cto.com

• ITeye

  • Java by
• Blog garden

• ChinaUnix

  • Partial Linux direction
• Open source Chinese community
• InfoQ
• The depth of the open source

• Bole online

  • Covering IT workplace, Web front end, back end, mobile end, database and other aspects of the content, technical side.
• ITPUB
• Tencent Cloud – Cloud + Community
• Ali cloud – cloud dwelling community
• IBM DeveloperWorks
• Developer Headlines
• LinkedKeeper

Abroad:

• DZone
• Reddit

Q&A and discussion communities

• segmentfault

  • Q&A + Column
• zhihu
• stackoverflow

Industry data analysis

• Iresearch net
• QUEST MOBILE
• National data
• TalkingData

The special website

• Testing:

  • The international lead test
  • Test nest
  • TesterHome

• Operations:

  (http://www.yunweipai.com/) * * [ops sent] [Abcdocker] (https://www.abcdocker.com/)

• Java:

  • ImportNew

    • Focus on Java technology sharing

  • HowToDoInJava

    • English blog

• security

  • Red and black alliance
  • FreeBuf

• Big data

  • Big Data in China

• Other special sites:

  • InfoQ

    • Emphasis on infrastructure, operation and maintenance direction

  • DockerInfo

    • Focusing on Docker applications as well as advice and tutorials

  • The Linux community

    • Linux Theme Community

Other classes

• Programmer Skills Graphic

Recommended reference book

Online e-books

• Deep Understanding of Spring Cloud and Microservice Construction
• Ali Technology Reference Atlas – Research and Development
• Ali Technical Reference Atlas – Algorithms
• 2018 Meituan Review Technology New Year (Collection) 70M
• InfoQ The Architect Monthly
• The Path of the Architect

Paper books

For more architecture books, see Awesome – Java-Books

development

• “Alibaba Java Development Manual” details

Architectural aspects

• The 12 Practices for a Software Architect: Technical Skills
• Details of the beauty of architecture
• Details of Distributed Services Architecture
• Talk about architecture
• Details of Cloud Native Application Architecture Practices
• Details of the core technology of the website architecture of hundred million traffic
• Details of “The Decade of Taobao Technology”
• Details of “The Way of Enterprise IT Architecture Transformation – Strategic Thought and Architecture Practice of China and Taiwan”
• Details of High Availability Architecture (Volume 1)

Technical Management

• Details of “CTO Talk”
• “Top of Technology Management” details
• Details of netease 1000 Nights: Internet Product Project Management Practics

The basic theory

• Details of the beauty of mathematics
• Details of “Programming Beguet”

tools

TODO

Big data

Technical resources

Open source resources

• github
• Apache Software Foundation

Manuals, documentation, tutorials

Domestic:

• W3Cschool

• Runoob.com

  • HTML, CSS, XML, Java, Python, PHP, design patterns and other introductory manuals.

• Love2.io

  • Many, many Chinese online e-books, is a new open source technology document sharing platform.

• gitbook.cn

  • Paid e-books.

• ApacheCN

  • AI, big data series of Chinese documents.

Abroad:

• Quick Code

  • Free online technical tutorials.

• gitbook.com

  • There are some Chinese e-books.

• Cheatography

  • Cheat Sheets, a single page document website.

• Tutorialspoint

  • Famous tutorial website, provide Java, Python, JS, SQL, big data and other high-quality introductory tutorials.

• LeetCode

  • Well-known test bank website, provide Java, Python, C#, C++, algorithm, SQL, and other high quality test banks and solutions.

Online classroom

• The apprentice is secured
• Geek time
• segmentfault
• Stark college
• Cattle from
• Institute of geeks
• 51 cto college

Meetings and activities

• QCon
• ArchSummit
• GITC Global Internet Technology Conference

Event Release Platform:

• Active line

Commonly used APP

• Geek time
• get

Looking for a job

• Boss hired straight
• Pull hook net
• cooperated
• 100Offer

tool

• Geek search

  • Technical article search engine.

Managed code

• Coding
• Yards cloud

File service

• Seven cattle
• And take the cloud

Integrated Cloud Service Provider

• Ali cloud
• Tencent cloud
• Baidu cloud
• Sina cloud
• Kingsoft cloud group
• Amazon Cloud (AWS)
• Google cloud
• Microsoft’s cloud

VPS

• Linode
• DigitalOcean
• Vultr