The 1.52.1 release of Rust addresses a bug in incremental compilation. The following is an explanation of the issue from the Rust team & translation of the improvement plan. Enjoy (sic: blog.rust-lang.org/2021/05/10/…

The Rust team has prepared a new version, 1.52.1, that addresses an error in incremental compilation that was made into a compiler error in 1.52.0. We recommend that all Rust users, including those currently using stable versions prior to 1.52.0, upgrade to 1.52.1 or disable incremental compilation. Guidance on how to do this is provided below.

If you have installed previous versions of Rust through Rustup, getting Rust 1.52.1 is easy:

rustup update stable
Copy the code

If you don’t already have one, you can get Rustup from the appropriate page on our website.

Abstract

This release addresses build issues in 1.52.0 that were caused by the addition of validation. The errors detected by this validation are present in all Rust versions 1 and may cause incorrect compilation in incremental builds, so demoting to a previously stable version is not a fix.

Users are encouraged to upgrade to 1.52.1 and, if earlier, to disable incremental compilation in their local environment: see the What You Should Do section for details on how to do this.

Incremental compilation for release versions is off by default, so few production versions will be affected (only for selected users).

Errors in incremental compilation can produce incorrect code in the final product, essentially malformed binaries, which means that in theory any behavior is possible. In practice, we only know there is a specific error compiling, but due to the incremental compilation errors are notoriously difficult to track: if the user from their binary file to see the result of the accident, they often just rebuild after some simple editing, this often leads to recompile enough to fix the errors.

This article will.

  1. Explain what these errors look like.
  2. Explain the role of inspection at a high level.
  3. Explain how this check is presented in Rust 1.52.0.
  4. Here’s what you should do if you see an unstable fingerprint on your project.
  5. Describe our plan for how Rust will address the issues discussed here.

What does the mistake look like?

The error message looks like this, and the key part is the text that says “unstable fingerprint found”.

thread 'rustc' panicked at 'assertion failed: `(left == right)`
  left: `Some(Fingerprint(4565771098143344972, 7869445775526300234))`,
  right: `Some(Fingerprint(14934403843752251060, 623484215826468126))`: found unstable fingerprints for <massive text describing rustc internals elided>

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.
Copy the code

This is an error caused by an internal consistency check, which, as the diagnosis notes, produces an “internal compiler error” (or ICE). In other words, it represents an error within the Rust compiler itself. In _ this _ case, ICE reveals an error in the incremental compilation that predates the 1.52.0 version and, if undetected, could result in an incorrect compilation.

What is a fingerprint? Why should we check them?

The Rust compiler supports “incremental compilation,” which was described in a 2016 blog post. When incremental compilation is enabled, the compiler breaks the input source into blocks and tracks how these input blocks affect the final build product. Then, when the input changes, it detects this, reuses previously built artifacts, and strives to build only those parts that need to respond to changes in the input source code.

Fingerprints are part of our architecture for detecting changes in input. More specifically, a fingerprint (along with some other states to establish context) is a 128-bit value designed to uniquely identify internal values used within the compiler. Some compiler internal results are stored on disk (” cached “) during runtime. The fingerprint is used to verify that the results of the new calculation are unchanged from the cached results. More details on this can be found in the relevant section of the USTC Development Guide).

Fingerprint stability check is a guarantee to assert the internal consistency of fingerprints. Sometimes the compiler is forced to rerun a query and expect the output to be the same as the previous incremental compilation session. The newly enabled validation checks that the value is indeed as expected, rather than assuming it is. In some cases, this is not the case due to errors in the compiler implementation.

history

We originally used these fingerprint checks in 2017 as a tool to develop RUSTC itself. It is only available through an unstable -Z flag and is only available for night and development versions.

Most recently, in March, we encountered an incorrect compilation that caused us to turn verify-ICH on by default. The Rust compiler team decided that rather than allow potential bad compilation (and subsequent bad behavior) to creep into Rust programmers’ binaries, it would be better to catch the fingerprint issue and halt the compilation.

When we first turned on fingerprint checking by default, there was a steady stream of issues submitted by users of the overnight (and beta) toolchain, and steady progress was made in identifying fixes, some of which have already landed.

Over the past week, we’ve started planning to improve the user experience so that the diagnosis that comes out of the test tells the programmer what to do. Unfortunately, this assumes that the new validation will ship with version 1.53, not version 1.52.

As it turns out, Verify-ICH was enabled in the recent 1.52.0 release.

Today’s new release, 1.52.1, addresses the damage caused by the newly added validation by temporarily changing the default value of the Rust compiler to disable increments unless the user intentionally chooses to.

How does this manifest itself

Basically, for certain sections, certain sequences of edit-compile loops will cause RUSTC to achieve “unstable fingerprints” of ICE. I showed an example at the beginning of this blog post.

Another recent example goes like this.

thread 'rustc' panicked at 'found unstable fingerprints for predicates_of(<massive text describing rustc internals elided>)', /rustc/... /compiler/rustc_query_system/src/query/plumbing.rs:593:5Copy the code

They are both inconsistent when comparing the incremental compilation cache stored on disk with the value computed during the current ruSTC call, which means that they are all generated using incremental compilation.

There are several ways you can turn on incremental compilation.

  1. You may be usingdev 或test The configuration fileTo build, theseThe configuration fileIncremental compilation is enabled by default.
  2. You may have already set itThe environment variable CARGO_INCREMENTAL=1
  3. You may beIn your Cargo configurationTo enable thebuild.incremental Set up the
  4. You may beIn the Cargo. Toml forA configuration file is enabledincrementalSettings.

If your project does not adjust the default values, then when running Cargo build –release, or when increments are disabled 1 on all Rust versions in the release configuration file, these issues should not affect your release build.

What should Rust programmers do

The internal compiler error requires you to report an error, and if you can do so, we still want to get that information. We want to know about the failures.

However, whether you submit a bug or not, the problem can be solved on your side in two ways.

  1. Upgrade to 1.52.1, if you have not already done so (which will invalidate your increments), or
  2. Remove your incremental compile cache (for example, by runningcargo clean), or
  3. By setting it in your environmentCARGO_INCREMENTAL=0Or in thebuild.incremental 到false config.tomlForce incremental compilation to be disabled.

We recommend 1.52.0 users upgrade to 1.52.1, which can disable incremental compilation.

We do not recommend 1.52.0 users downgrade to an earlier version of Rust to deal with this issue. As mentioned above, there is at least one example of a silent incorrect compilation caused by incremental compilation that went undetected until we added fingerprint checks.

If users are willing to handle the issue of incremental verification ICE and want to opt back to 1.52.0 behavior, they can put RUSTC_FORCE_INCREMENTAL, 1 in their environment. This way the Rust compiler will respect the -cincremental option passed by Cargo, and things will proceed as before, but with added validation. Note that this flag will not enable increments (whether by Cargo or otherwise) if increments are not enabled separately.

If you are using a pre-1.52.0 tool chain and want to continue to do so, we recommend that you disable incremental compilation to avoid silent error compilation.

In all versions of Rust that have been incretized and logged in, it has made a big difference in build times for many users, and this improvement will only get bigger over time. We acknowledge that the workarounds and proposals presented here are painful and will work to ensure that the situation is as temporary as possible.

What does Rust do to solve this problem

Short-term plans

We released 1.52.1 today.

  • Disable incremental compilation in the Rust compiler (unless required by a new environment variable,RUSTC_FORCE_INCREMENTAL=1).
  • If incremental compilation is enabled, the diagnostic output for the new validation is improved, indicating how these errors can be resolved by either clearing the delta state or disabling the delta.

This is intended to be a mitigation measure to help most Rust users have a path to upgrade to a secure Rust compiler that does not run the risk of incorrectly compiling their code, but also provides an option for users willing to handle errors.

We expect to continue to actively invest in fixing these bugs, and depending on our confidence in the fix, we may release a 1.52.2 point version that ports these fixes to the stable channel. Users who want to help us test can use the nightly channel and report any ICE errors they see to Rust-lang/Rust.

We do not currently intend to disable increments on test channels, but this decision has not been firmly committed. There are some fixes in beta 1.53 today, so users who want to continue using increments may want to switch to this version. And of course, Nightly is always updated.

A long-term plan

The long term plan is to fix bugs! Incremental compilation is the only realistic way the Rust compiler can provide a fast edit-compile-run cycle for all programmers, so we need to address all of the issues identified so far with Verify-ICH. So far, there are 32 such questions, although many are repetitive).

We are actively investing in this, and some of the errors have been identified and fixed. Depending on the fix, incremental compilation may be re-enabled in future stable releases (1.53 and beyond).

The Rust team will also develop plans to ensure that we have a better bug tracking system in the future, both to prevent situations like this from happening again and to track bugs more accurately as they propagate across channels, further improving the stability of our releases.

Since increments were first enabled, which is in Rust 1.24.