By Wu Xiangxiang @Pymongo/Edited by Zhang Handong

原文: custom Rust lint

Requirement: TFN of vscode-ra will generate fn feature function, I hope the static analysis can help me check it out, so that the feature function will not be submitted to Github

Feasibility of static analysis

Since Intellij-Rust is written in Kotlin, the advantage is that it does not depend on RustC, while the disadvantage is that the source code of RustC cannot be analyzed for the time being

Because static analysis/procedure macro/compiler principle related research requires in-depth study of compiler source code, SO I only consider using Rust static analysis Rust code solution, not considering Intellij-Rust

  • Procedural macros: Functions marked by procedural macros can be performed AST static analysis at compile time, but having to mark each function is inconvenient
  • Change rustc source code: for example non_ascii_idents lint, but rustc compilation is too slow
  • Cargo Clippy: You can change the source code of Clippycargo installCompile to cargo subcommand
  • Cargo DyLint: Feasibility pending

Lint means a lot to companies

For example, the company team forbids the use of recursive calls in project code because Rust has limited optimization for nonlinear recursion, and poor recursion can cause stack explosion and production server panic

After all, there are many implementations of Rust that use recursion, so it’s impossible to mention PR to officially add “recursion-forbidden” lint to Clippy

If the company manually reviews recursive code, it is not only inefficient but can not guarantee 100% accuracy

By adding custom Lint to a company’s CI/CD process, you can automatically detect code that doesn’t conform to the company’s COding_style

Cargo – Lint executable file

To distinguish it from clippy’s executable file names, I made the following changes to the clippy code for fork:

diff --git a/Cargo.toml b/Cargo.toml index 9b5d9b2ad.. 17e13950d 100644 -- a/Cargo. Toml +++ b/Cargo. Toml @@-12,13 +12,15 @@ publish = false [[bin]] -name = "freight-clippy" +name = "cargo-lint" [[bin]] -name = "clippy-driver" +name = "lint-driver" [dependencies] diff --git a/src/main.rs b/src/main.rs index 7bb80b119.. 3df9e40d5 100644 -- -- a/ SRC /main.rs +++ b/ SRC /main.rs @@ -107,7 +107,7 @@ impl ClippyCmd {- .with_file_name("clippy-driver"); + .with_file_name("lint-driver");Copy the code

First clippy requires freight-clippy and Clippy-driver executables, so I installed Freight-Lint in the following way:

cargo install –bin=cargo-lint –bin=lint-driver –path=.

Adding a new Lint

Refer to the documentation for addling_lints

Cargo Dev is the Cargo Alias of the Clippy project, and new Lint can be created with cargo Dev

cargo dev new_lint –name=my_lint_function_name_is_feature –pass=early –category=correctness

diff --git a/clippy_lints/src/lib.rs b/clippy_lints/src/lib.rs index f5082468a.. 005f99895 100644 -- a/clippy_lints/ SRC /lib.rs +++ b/clippy_lints/ SRC /lib.rs @@-276,6 +276,7 @@mod mut_mutex_lock; mod mutex_atomic; +mod my_lint_function_name_is_feature; mod needless_arbitrary_self_type; @@-822,6 +823,7 @@pub fn register_plugins(store: &mut rustc_lint::LintStore, sess: &Session, conf: mutex_atomic::MUTEX_INTEGER, + my_lint_function_name_is_feature::my_lint_function_name_is_feature, @@-1345,6 +1347,7 @@pub fn register_plugins(store: &mut rustc_lint::LintStore, sess: &Session, conf: LintId::of(mutex_atomic::MUTEX_ATOMIC), + LintId::of(my_lint_function_name_is_feature::my_lint_function_name_is_feature), @@-1702 6 +1705 7 @@pub fn register_plugins(store: &mut rustc_lint::LintStore, sess: &Session, conf: LintId::of(mut_key::MUTABLE_KEY_TYPE), + LintId::of(my_lint_function_name_is_feature::my_lint_function_name_is_feature),Copy the code

Two files have been added:

  1. clippy_lints/src/my_lint_function_name_is_feature.rs
  2. tests/ui/my_lint_function_name_is_feature.rs

Changed a file: clippy_lints/ SRC /lib.rs

However, this would change changelo.md and clippy_lints/ SRC /lib.rs, making it difficult for me to incorporate the changes made to Clippy upstream

Lint unit tests

I’ll name my Lint: my_lint_function_name_is_feature

Therefore, the key point of unit testing is that errors will be reported only if the function name is feature, and not if the variable name is feature

TESTNAME=my_lint_function_name_is_feature cargo uitest

Cargo dev Bless generates a my_lint_function_name_is_feature.stderr file from the last cargo Uitest run

Lint: clippy::my_lint_function_name_is_feature

Unknown lint

How Clippy Works and Syncing changes between Clippy and rust-lang/ Rust

The section notes that Clippy seems to be strongly bound to the RustC version, so you have to wait until Rust updates the Clippy subrepository for the new Lint to take effect?

Lint nonstandard_macro_braces (2021-06-19) Unknown Lint

After reading the document several times for various errors while adding a new Version of Lint to Clippy, I turned to a more feasible static analysis solution

cargo dylint

Due to Rust custom lint static analysis inspection data are very few, I search so can only find this article: www.trailofbits.com/post/write-…

Fortunately cargo DyLint is written in exactly the same way as the new Version of Clippy/Rustc, and the rustc study materials are sufficient to learn dyLint

First you need to install dyLint and dyLint’s linker:

cargo install cargo-dylint dylint-link

Dylint template

Although DyLint is very similar to Clippy, it is recommended to follow the DyLint example

  1. dylint-template
  2. Dylint code repository examples for each example
  3. I modified the template based on path_separator: github.com/pymongo/my_…

⚠ Note that since dyLint works like Clippy, versions of dylint/rustc/clippy_utils must be compatible with each other

Changing the Rustc or Clippy version will likely cause DyLint to fail or not take effect. Don’t change dependent versions!

Dylint runs methods

Let’s say our custom Lint source directory is MY_LINTS_PATH

export MY_LINTS_PATH=/home/w/repos/my_repos/my_lints

Suppose the path of the company project code is /home/w/temp.other_rust_project

⚠ attention! If the new version of Lint does not work, then cargo clean MY_LINTS_PATH and recompile it

¶ 1. Run dyLint in the folder where you wrote Lint

cd $MY_LINTS_PATH

cargo dylint –all — –manifest-path=/home/w/temp/other_rust_project/Cargo.toml

¶ 2. Import in the project folder

DYLINT_LIBRARY_PATH=$MY_LINTS_PATH/target/debug cargo dylint –all

¶ 3. Package.metadata. dyLint in project Cargo. Toml

Warning: No libraries were found

Early/Late Lint concept

See Overview of the Compiler-Rustc-dev-Guide

I’ll roughly summarize the Rust compilation process as follows:

(rustc_args_and_env -rustc_driver-> rustc_interface::Config)

  1. source_code_text(bytes) -rustc_lexer-> TokenStream
  2. TokenStream -rustc_parse-> AST
  3. AST analysis: macro_expand, name_resolution, feature_gating, checking/early_lint
  4. AST convert to HIR
  5. HIR analysis: type/trait checking, late_lint
  6. HIR convert to MIR
  7. MIR analysis: ownership/lifetime/borrow checking
  8. MIR Optimizations
  9. MIR convert to LLVM IR
  10. LLVM backend compile LLVM IR to executable or library

So early_Lint analyzes AST code, late_lint analyzes HIR code

The macro and procedure macro are the input token_stream, and the macro output is the expanded token_stream (see heapsize Procedure macro).

Dylint Adds a new Lint

If your company needs a Version of Lint, any function with the name todo should throw a warning

§ fn_name_contains_todo-step_1: Create a Lint file

I’ll name this version of Lint fn_name_contains_todo. Be sure that the name of lint does not conflict with rustc’s version of Lint

Git Clone github.com/pymongo/my_…

Then add a new fn_name_contains_todo.rs file in SRC /

And under SRC /lib.rs add mod fn_name_contains_todo; Add a new file to the module tree

§ fn_name_contains_todo-step_2: lint implementation

First define the lint structures and constants in fn_name_contains_todo.rs, exactly the same way clippy creates a new Lint

rustc_session::declare_lint! {
    pub FN_NAME_CONTAINS_TODO,
    Warn,
    "fn_name_contains_todo"} rustc_session::declare_lint_pass! (FnNameContainsTodo => [FN_NAME_CONTAINS_TODO]);Copy the code

Since analyzing variable or function names only requires an AST, not an HIR with type information, use Early Lint only

impl rustc_lint::EarlyLintPass for FnNameContainsTodo {
    fn check_fn(&mut self,
        cx: &rustc_lint::EarlyContext<'_>,
        fn_kind: rustc_ast::visit::FnKind<'_>,
        span: rustc_span::Span,
        _: rustc_ast::NodeId,
    ) {
        // Ignore FnKind::Closure
        if let rustc_ast::visit::FnKind::Fn(_, ident, ..) = fn_kind {
            if ident.as_str().contains("todo") {
                clippy_utils::diagnostics::span_lint(
                    cx,
                    FN_NAME_CONTAINS_TODO,
                    span,
                    "fn name with todo is not allow to commit",); }}}}Copy the code

§ fn_name_contains_todo-step_3: Register Lint

After the implementation of Lint is written, it is in the pub fn register_lints of lib.rs

  1. Add fn_name_containS_todo :: fn_name_contains_todo to the incoming parameter group of lint_store.register_lints
  2. Join lint_store early_pass line. Register_early_pass (| | Box: : the new (fn_name_contains_todo: : FnNameContainsTodo));

Lint adds register_early_pass if it uses EarlyLintPass, and register_late_pass if it uses LateLIntPass

⚠ Note: lint_store.register_lints is added to the Lint constant, and lint_store.register_early_pass is passed to the Lint structure

After recompiling the code, the new version of Lint takes effect

§ fn_name_contains_todo-Step_4: (Optional) Test Lint

As a crude test, we added the following to SRC /lib.rs:

#[allow(dead_code)]
fn todo() {}Copy the code

Then run dyLint to analyze the current project (i.e., the Lint source project):

cargo clean && cargo b && cargo dylint –all

Checking my_lints v0.1.0 (/home/w/repos/my_repos/my_lints) Warning: Checking my_lints v0.1.0 (/home/w/repos/my_repos/my_lints) Warning: fn name with todo is not allow to commit --> src/lib.rs:26:1 | 26 | / fn todo() { 27 | | 28 | | } | |_^ | = note: `#[warn(fn_name_contains_todo)]` on by default warning: 1 warning emittedCopy the code

If you feel that lib.rs other code is polluting cargo expand or TokenSteam/HIR code expansion

For example, if I just want to see HIR code for two functions, I can create a examples/fn_name_contains_todo.rs file or use –tests to specify a static analysis file

For example cargo clean && cargo b && cargo dylint –all — –examples

As for UI testing methods, I recommend reading the Clippy documentation rather than demonstrating it in this article

⚠ attention! : Do not add RUST_LOG=info to the environment variable at the log level when you run the UI test. Otherwise, logs will be output and test comparison fails

Lint disables recursive code

Since I am not very familiar with AST/HIR parsing, I refer to the following code:

  • Rustc_lint: : builtin: : UNCONDITIONAL_RECURSION: didn’t find the implementation code (as in MIR)
  • Clippy main_recursion of lint

It’s not hard to write code that statically detects recursion, like main_recursion:

impl rustc_lint::LateLintPass<'_> for MyLintRecursiveCode {
    fn check_expr_post(&mut self, cx: &rustc_lint::LateContext<'_>, expr: &rustc_hir::Expr< '_>) {
        if let rustc_hir::ExprKind::Call(func_expr, _) = &expr.kind {
            Function call func_expr ** stack frame ** function defid
            let func_expr_owner_defid = func_expr.hir_id.owner.to_def_id();
            if let rustc_hir::ExprKind::Path(rustc_hir::QPath::Resolved(_, path)) = &func_expr.kind {
                // path.res: The resolution of a path or export
                if let Some(func_expr_call_defid) = path.res.opt_def_id() {
                    if func_expr_owner_defid == func_expr_call_defid {
                        clippy_utils::diagnostics::span_lint(
                            cx,
                            MY_LINT_RECURSIVE_CODE,
                            expr.span,
                            "our company forbid recursive code, Reference: company_code_style.pdf, page 17",); } } } } } }Copy the code

Unfortunately, it is not possible to detect the infinite recursion caused by a calling B and B calling A

The official Rust issue 57965, 70727 also discusses how to detect infinite recursion across function calls at compile time

More Lint requirements

f32_cast_to_f64

The company business has a price parameter that needs to retain two decimal places, and then convert from F32 to F64 for transmission, which is the JavaScript Number type

However, when f32 with two decimal places is converted to F64, the accuracy will be lost: 0.1_F32 as F64 = 0.10000000149011612

But this is normal IEEE behavior for floating-point numbers, and the same problem occurs with float -> double conversions in C/C++

Lint currently only has size comparisons, I32 as F32, and other Rust floating-point numbers that Lint does not have a business need for accuracy loss detection

So it’s important to customize some of the floating-point lint for your business to avoid exceptions to the floating-point numbers shown on the front end

conclusion

Benefit from the powerful debug macro DBG! And AST/HIR excellent structure design, such as the author of a non-computer major without a compiler principles course level can easily customize static analysis

The source repository for this article: github.com/pymongo/my_… In which you are welcome to contribute requirements or ideas for code reviews