Rollup rollup rollup rollup

Hi, I’m Xiaoyu Xiaoyu, dedicated to sharing interesting and practical technical articles. Content is divided into translation and original, if you have any questions, feel free to comment or private letter, I hope to progress with you. Sharing is not easy, hope to get your support and attention.

plan

The Rollup series is intended to be chapter by chapter, more concise, more focused and easier to understand

It is currently intended to be divided into the following chapters:

Rollup. rollup <==== current article
rollup.generate + rollup.write
rollup.watch
tree shaking
plugins

TL; DR

Rollup.rollup () rollup.rollup() rollup.rollup() rollup.rollup() rollup.rollup() rollup.rollup()

Configure collection and standardization
File analysis
Source code compilation, generate AST
Module to generate
Depend on the resolution
Filtration purification
Output chunks

The idea is simple, but the details are complicated. However, we do not have to focus on specific implementation, after all, all roads lead to Rome, we can absorb and improve or learn some code techniques or optimization methods, in my opinion, this is a good way to read the source code. 🙂

Pay attention to the point

All the notes are here and can be read by yourself

!!!!!!!!! Version => Rollup version read by the author is: 1.32.0

!!!!!!!!! Tip => labeled TODO for specific implementation details, will be analyzed as the case may be.

!!!!!!!!! Note that => each subtitle is an internal implementation of the parent title (function)

!!!!!!!!! We can return the id of the file that we want to return (i.e., the address, relative path, and decision path) for rollup to load

Rollup is a core that only does the most basic things, such as providing a default loading mechanism for modules (files), such as packaging different styles of content, our plug-in provides a path for loading files, parsing file contents (handling TS, Sass, etc.), and so on. It is a pluggable design. Similar to Webpack, plug and pull is a very flexible and long-term iterative design, which is also the core of a large framework, there is power in numbers

Main generic modules and their meanings

Graph: a globally unique Graph containing entries and dependencies, operations, caches, etc. Is the heart of rollup
PathTracker: The side-effect-free module relies on path tracking
PluginDriver: Plug-in driver, call plug-in, provide plug-in environment context, etc
FileEmitter: Resource operator
GlobalScope: GlobalScope, as opposed to local scope
ModuleLoader: ModuleLoader
NodeBase: The construction base class of the AST syntax (ArrayExpression, AwaitExpression, etc.)

Main process analysis

1. Call getInputOptions to standardize input parameters

const inputOptions = getInputOptions(rawInputOptions);
Copy the code

1.1. Call mergeOptions to set the default input and Output configurations and return error messages for input configuration and using illegal configuration properties
```
let { inputOptions, optionError } = mergeOptions({
  config: rawInputOptions
});
Copy the code
```
1.2. Call the Options hook function to make custom changes before the input match is fully standardized
```
inputOptions = inputOptions.plugins! .reduce(applyOptionHook, inputOptions);Copy the code
```
1.3. Standardized plug-in operation: Set the default plug-in name for the plug-in that does not have the name attribute in the return object => at Position Index value of the current plug-in among all plug-ins
```
inputOptions.plugins = normalizePlugins(inputOptions.plugins! , ANONYMOUS_PLUGIN_PREFIX);Copy the code
```

1.4. Incompatible embeddingDynamic introduction moduleOr keep the configuration of the module and report an error

/ / will dynamic import dependence (import | the require. Ensure () | other) are embedded into a chunk and not create a separate package, relevant code logic is as follows
if (inputOptions.inlineDynamicImports) {
  // preserveModules: preserveModules as much as possible, rather than mixing them up and creating fewer chunks, default to false and not open
  if (inputOptions.preserveModules) // If it is enabled, it conflicts with embedding
    return error({
      code: 'INVALID_OPTION'.message: `"preserveModules" does not support the "inlineDynamicImports" option.`
    });
  // For other judgments, refer to the code repository: index.ts
} else if (inputOptions.preserveModules) {
  // Reject functionality named after the original file
  if (inputOptions.manualChunks)
    return error({
      code: 'INVALID_OPTION'.message: '"preserveModules" does not support the "manualChunks" option.'
    });
  // For other judgments, refer to the code repository: index.ts
}
Copy the code

1.5. Return the processed INPUT configuration
```
return inputOptions;
Copy the code
```

2. Check whether the performance check function is enabled to check the inputOptions. Perf property
```
initialiseTimers(inputOptions);
Copy the code
```

3. Create a diagram with input configuration and watch parameters. Watch is not currently considered

const graph = new Graph(inputOptions, curWatcher);
Copy the code

3.1. Initialize the warning function and cache the warning that has been prompted

this.onwarn = (options.onwarn as WarningHandler) || makeOnwarn();
Copy the code

3.2. Mount the path tracking system to the current graph. No constructors, only properties and methods to change properties
```
this.deoptimizationTracker = new PathTracker();
Copy the code
```
3.3. Initialize the unique module cache container of the current graph, and assign the cache attribute of the last packing result to the next packing to improve the packing speed =>
```
  this.cachedModules = new Map(a);Copy the code
```

3.4. Read modules and plug-ins from the last build result passed. Plug-in cache reference =>, explained below.

if (options.cache) {
  if (options.cache.modules)
    for (const module of options.cache.modules) this.cachedModules.set(module.id, module);
}

if(options.cache ! = =false) {
  this.pluginCache = (options.cache && options.cache.plugins) || Object.create(null);

  for (const name in this.pluginCache) {
    const cache = this.pluginCache[name];
    for (const key of Object.keys(cache)) cache[key][0] + +; }}Copy the code

3.5. Treeshake information mount.

if(options.treeshake ! = =false) {
  this.treeshakingOptions = options.treeshake && options.treeshake ! = =true
      ? {
          annotations: options.treeshake.annotations ! = =false.moduleSideEffects: options.treeshake.moduleSideEffects,
          propertyReadSideEffects: options.treeshake.propertyReadSideEffects ! = =false.pureExternalModules: options.treeshake.pureExternalModules,
          tryCatchDeoptimization: options.treeshake.tryCatchDeoptimization ! = =false.unknownGlobalSideEffects: options.treeshake.unknownGlobalSideEffects ! = =false}, {annotations: true.moduleSideEffects: true.propertyReadSideEffects: true.tryCatchDeoptimization: true.unknownGlobalSideEffects: true
        };
  if (typeof this.treeshakingOptions.pureExternalModules ! = ='undefined') {
    this.warnDeprecation(
      `The "treeshake.pureExternalModules" option is deprecated. The "treeshake.moduleSideEffects" option should be used instead. "treeshake.pureExternalModules: true" is equivalent to "treeshake.moduleSideEffects: 'no-external'"`.false); }}Copy the code

3.6. Initialize the code parser and refer to graph.ts for specific parameters and plug-ins

this.contextParse = (code: string, options: acorn.Options = {}) = >
  this.acornParser.parse(code, { ... defaultAcornOptions, ... options, ... this.acornOptions })as any;
Copy the code

3.7. Plug-in drives

this.pluginDriver = new PluginDriver(
  this, options.plugins! .this.pluginCache,
  // Whether the address of soft connection is used as the context when processing soft connection files. False indicates yes, and true indicates no.
  options.preserveSymlinks === true,
  watcher
);
Copy the code

3.7.1. Deprecate API warnings and mount parameters

3.7.2. Instantiate FileEmitter and set the methods carried by the instance to the plug-in driver

// basePluginDriver is the sixth parameter to PluginDriver and represents the 'root' plug-in driver for graph
this.fileEmitter = new FileEmitter(graph, basePluginDriver && basePluginDriver.fileEmitter);
this.emitFile = this.fileEmitter.emitFile;
this.getFileName = this.fileEmitter.getFileName;
this.finaliseAssets = this.fileEmitter.assertAssetsFinalized;
this.setOutputBundle = this.fileEmitter.setOutputBundle;
Copy the code

3.7.3. Plug-in splicing

this.plugins = userPlugins.concat(
  basePluginDriver ? basePluginDriver.plugins : [getRollupDefaultPlugin(preserveSymlinks)] 
);
Copy the code

3.7.4. Cache the context of the plug-in, which will be retrieved by index and injected into the plug-in when executing the plug-in

// Use map to inject a plugin-specific context into each plug-in and cache it
this.pluginContexts = this.plugins.map(
  getPluginContexts(pluginCache, graph, this.fileEmitter, watcher)
);
Copy the code

3.7.5. An error is reported when input and output Settings conflict

if (basePluginDriver) {
  for (const plugin of userPlugins) {
    for (const hook of basePluginDriver.previousHooks) {
      if (hook inplugin) { graph.warn(errInputHookInOutputPlugin(plugin.name, hook)); }}}}Copy the code

3.8. Setting of monitoring mode

	if (watcher) {
		const handleChange = (id: string) = > this.pluginDriver.hookSeqSync('watchChange', [id]);
		watcher.on('change', handleChange);
		watcher.once('restart', () => {
			watcher.removeListener('change', handleChange);
		});
	}
Copy the code

3.9. Global Context

this.scope = new GlobalScope();
Copy the code

3.10. Set the global context of the module. Default is false

this.context = String(options.context);

	// Whether the user has customized the context
	const optionsModuleContext = options.moduleContext;
	if (typeof optionsModuleContext === 'function') {
		this.getModuleContext = id= > optionsModuleContext(id) || this.context;
	} else if (typeof optionsModuleContext === 'object') {
		const moduleContext = new Map(a);for (const key in optionsModuleContext) {
			moduleContext.set(resolve(key), optionsModuleContext[key]);
		}
		this.getModuleContext = id= > moduleContext.get(id) || this.context;
	} else {
		this.getModuleContext = (a)= > this.context;
	}
Copy the code

3.11. Initialize moduleLoader for module (file) parsing and loading

// Module (file) parse load, internal call resolveID and load hook, so that users have more operation power
this.moduleLoader = new ModuleLoader(
		this.this.moduleById,
		this.pluginDriver, options.external! , (typeof options.manualChunks === 'function' && options.manualChunks) as GetManualChunk | null,
		(this.treeshakingOptions ? this.treeshakingOptions.moduleSideEffects : null)! , (this.treeshakingOptions ? this.treeshakingOptions.pureExternalModules : false)! ;Copy the code

4. Execute the buildStart hook function to package the chunks for subsequent builds and writes

try {
  	// the buildStart hook function fires
  	await graph.pluginDriver.hookParallel('buildStart', [inputOptions]);
  	// In this step, through the id, the topology relationship is analyzed in depth, and the useless chunks are removed to generate our chunks
  
  // Build logic is detailed below
  	chunks = await graph.build( // This is a "chunks" closure, so generate and write can be used
  		inputOptions.input asstring | string[] | Record<string, string>, inputOptions.manualChunks, inputOptions.inlineDynamicImports! ) ; }catch (err) {
  	const watchFiles = Object.keys(graph.watchFiles);
  	if (watchFiles.length > 0) {
  		err.watchFiles = watchFiles;
  	}
  	await graph.pluginDriver.hookParallel('buildEnd', [err]);
  	throw err;
  }
Copy the code

5. Return an object, including cache, listen file, and generate, write methods
```
return {
  cache,
  watchFiles,
  generate,
  write
}
Copy the code
```

Graph.build logical parsing

The Build method deeply analyzes the topological relationship through ID, removes useless chunks, and generates our chunks accept three parameters: entry, extracting common chunks rules (manualChunks), and whether to embed dynamic import modules

Build is a very simple method, which is to produce our chunks. He returns a Promise object for later use.

  return Promise.all([entry module,/ / code for: this. ModuleLoader. AddEntryModules (normalizeEntryModules (entryModules), true)User-defined public modules// There is no return value for this block, but the public module is cached on the module loader, and the result is returned by the entry module agent. Clever way to kill two birds with one stone
  ]).then((The return of the entry module) = > {
    // Module dependency handling
    return chunks;
  });
Copy the code

Entry module: enclosing moduleLoader. AddEntryModules (normalizeEntryModules (entryModules), true)

NormalizeEntryModules normalize entries to return a uniform format:

  UnresolvedModule {
      fileName: string | null;
      id: string;
      name: string | null;
  }
Copy the code

AddEntryModules loads, reloads, sorts modules, and finally returns modules, common chunks. The modulesById(Map object) of ModuleLoaders is cached during the loading process. Part of the code is as follows:

  // Module loading part
  private fetchModule(
    id: string,
    importer: string,
    moduleSideEffects: boolean,
    syntheticNamedExports: boolean,
    isEntry: boolean
  ): Promise<Module> {
    // The main process is as follows:
    
    // Get cache, improve efficiency:
    const existingModule = this.modulesById.get(id);
    if (existingModule instanceof Module) {
      existingModule.isEntryPoint = existingModule.isEntryPoint || isEntry;
      return Promise.resolve(existingModule);
    }
    
    // Create a module:
    const module: Module = new Module(
      this.graph,
      id,
      moduleSideEffects,
      syntheticNamedExports,
      isEntry
    );
    
    // Cache for optimization
    this.modulesById.set(id, module);
    
    // Set the listener for each inbound module
    this.graph.watchFiles[id] = true;
    
    // Call the user-defined manualChunk method to get the common chunks alias, for example:
    / / such as manualChunkAlias (id) {
    // if (xxx) {
    // return 'vendor';
    / /}
    // }
    const manualChunkAlias = this.getManualChunk(id);
    
    // Cache to manualChunkModules
    if (typeof manualChunkAlias === 'string') {
      this.addModuleToManualChunk(manualChunkAlias, module);
    }
    
    // Call the load hook function and return the result of processing, where the second array argument is the argument passed to the hook function
    return Promise.resolve(this.pluginDriver.hookFirst('load', [id]))
      .cache()
      .then(source= > {
        // Unified format: sourceDescription
        return {
          code: souce,
          // ...
        }
      })
      .then(sourceDescription= > {
        // Return the code processed by the transform hook function, such as JSX parsing, ts parsing
        Reference: / / https://github.com/rollup/plugins/blob/e7a9e4a516d398cbbd1fa2b605610517d9161525/packages/wasm/src/index.js
        return transform(this.graph, sourceDescription, module);
      })
      .then(source= > {
        // The result of the code compilation hangs on the currently parsed entry module
        module.setSource(source);
        // The module ID is bound to the module
        this.modulesById.set(id, module);
        // Handle module dependencies and mount exported modules to module
        / /!!!!!! Note that the module created in fetchAllDependencies is created through the ExternalModule class with other entry modules
        return this.fetchAllDependencies(module).then((a)= > {
          for (const name in module.exports) {
            if(name ! = ='default') {
              module.exportsAll[name] = module.id; }}for (const source of module.exportAllSources) {
            const id = module.resolvedIds[source].id;
            const exportAllModule = this.modulesById.get(id);
            if (exportAllModule instanceof ExternalModule) continue;

            for (const name inexportAllModule! .exportsAll) {if (name in module.exportsAll) {
                this.graph.warn(errNamespaceConflict(name, module, exportAllModule!) ); }else {
                module.exportsAll[name] = exportAllModule! .exportsAll[name]; }}}// Return these processed Module objects, converted from the id(file path) to an object with nearly complete file information.
        return module; })}Copy the code

  / / to heavy
  let moduleIndex = firstEntryModuleIndex;
		for (const entryModule of entryModules) {
			// Whether it is user-defined. The default value is yes
			entryModule.isUserDefinedEntryPoint = entryModule.isUserDefinedEntryPoint || isUserDefined;
			const existingIndexModule = this.indexedEntryModules.find(
				indexedModule= > indexedModule.module.id === entryModule.id
			);
			// Perform entry de-weighting according to moduleIndex
			if(! existingIndexModule) {this.indexedEntryModules.push({ module: entryModule, index: moduleIndex });
			} else {
				existingIndexModule.index = Math.min(existingIndexModule.index, moduleIndex);
			}
			moduleIndex++;
		}
  / / sorting
  this.indexedEntryModules.sort(({ index: indexA }, { index: indexB }) = >
			indexA > indexB ? 1 : - 1
		);
Copy the code

The dependency handling part of the module

Modules that have been loaded and processed are cached in the moduleById, so they are directly traversed and classified according to their module classes

  // moduleById is the store of id => Module, which is all valid entry modules
		for (const module of this.moduleById.values()) {
			if (module instanceof Module) {
				this.modules.push(module);
			} else {
				this.externalModules.push(module); }}Copy the code

Get all the entry points, find the correct ones, remove useless dependencies, and filter out the modules that are really entry points

  // This. Link (entryModules) inside the method
  
  // Find all dependencies
  for (const module of this.modules) {
    module.linkDependencies();
  }
  
  // Returns the relative path of all entry boot modules (i.e., non-external modules) and modules that depend on a loop of results
  const { orderedModules, cyclePaths } = analyseModuleExecution(entryModules);
  
  // Warn against dead-loop paths
  for (const cyclePath of cyclePaths) {
    this.warn({
      code: 'CIRCULAR_DEPENDENCY'.cycle: cyclePath,
      importer: cyclePath[0].message: `Circular dependency: ${cyclePath.join('- >')}`
    });
  }
  
  // Filter out the real entry to start the module and assign it to modules
  this.modules = orderedModules;
  
  // Further parsing of ast syntax
  // TODO:Additional details as appropriate
  for (const module of this.modules) {
    module.bindReferences();
  }
  
Copy the code

The rest

  // Import all exports, set the correlation
  // TODO:Additional details as appropriate
    for (const module of entryModules) {
			module.includeAllExports();
		}
  
  // Set the context for the imported environment based on the user's Treeshaking configuration
		this.includeMarked(this.modules);
  
		// Check all unused modules for a prompt warning
		for (const externalModule of this.externalModules) externalModule.warnUnusedImports();
  
  // Add hash to each entry module for later consolidation into a chunk
  if (!this.preserveModules && ! inlineDynamicImports) { assignChunkColouringHashes(entryModules, manualChunkModulesByAlias); }let chunks: Chunk[] = [];
  
  // Create chunk for each module
		if (this.preserveModules) {
			// Iterate over the entry module
			for (const module of this.modules) {
				// Create a Chunk instance object
				const chunk = new Chunk(this[module]);
				// Is an entry module and is not empty
				if (module.isEntryPoint || ! chunk.isEmpty) { chunk.entryModules = [module]; } chunks.push(chunk); }}else {
			// Create as few chunks as possible
			const chunkModules: { [entryHashSum: string]: Module[] } = {};
			for (const module of this.modules) {
				// Convert the previously set hash value to string
				const entryPointsHashStr = Uint8ArrayToHexString(module.entryPointsHash);
				const curChunk = chunkModules[entryPointsHashStr];
				// If yes, add module, if not create and add, the same hash values will be added together
				if (curChunk) {
					curChunk.push(module);
				} else {
					chunkModules[entryPointsHashStr] = [module]; }}// Sort the chunks with the same hash value and add them to the chunks
			for (const entryHashSum in chunkModules) {
				const chunkModulesOrdered = chunkModules[entryHashSum];
				// This should represent the order in which it was introduced, or the order in which it was executed
				sortByExecutionOrder(chunkModulesOrdered);
				// Create a new chunk with the chunkModulesOrdered
				const chunk = new Chunk(this, chunkModulesOrdered); chunks.push(chunk); }}// Mount dependencies to each chunk
		for (const chunk of chunks) {
			chunk.link();
		}
Copy the code

This is the main flow analysis of rollup. Rollup. Refer to the code base comments for details

Specific analysis of some functions

Plugin caching capability resolution, which gives developers the ability to cache data on plug-ins by using the cacheKey to share data between different instances of the same plug-in

function createPluginCache(cache: SerializablePluginCache) :PluginCache {
	// Use closures to cache the cache
	return {
		has(id: string) {
			const item = cache[id];
			if(! item)return false;
			item[0] = 0; // If yes, then reset the access expiration times, so that the user intends to use actively
			return true;
		},
		get(id: string) {
			const item = cache[id];
			if(! item)return undefined;
			item[0] = 0; // If yes, reset the access expiration times
			return item[1];
		},
		set(id: string, value: any) {
			cache[id] = [0, value];
		},
		delete(id: string) {
			return deletecache[id]; }}; }Copy the code

You can see that rollup leverages the structure of the object addend group to provide caching capabilities for the plug-in:

{
  test: [0.'content']}Copy the code

The first item of the array is the current access counter, which is linked to the number of cache expiration times, plus the closure capabilities of JS to provide simple and practical caching capabilities on the plug-in

conclusion

So far, this has once again reinforced the importance of singleness and dependency injection, such as module loaders, plug-in drivers, and Graph. There is rollup modularity, webPack is similar, and VUE is similar in that it takes concrete content and turns it into abstract data, and then continues to mount other abstract data that depends on it, subject to certain specifications, such as estREE specifications.

I have always been interested in construction, nearly half of my Github is related to construction, so this time, from the rollup entrance, I started to uncover the layers of haze to build the world, and give us a clear world. 🙂

The rollup series does not refer to other people’s sharing. (no one has analyzed rollup yet.) I read line by line on my own, so inevitably there are some things that are not quite right. There is no way, reading other people’s code, some places are like guessing women’s mind, too damn hard, so if there is something wrong, I hope you can give more advice and learn from each other.

Or that sentence, creation is not easy, I hope to get everyone’s support, and you are encouraged, we see you next time!

plan

TL; DR

Pay attention to the point

Main generic modules and their meanings

Main process analysis

Graph.build logical parsing

Specific analysis of some functions

conclusion

Related Posts

Learn a vUE plugin (13) every day — HTML2Canvas

Animation weapon – Lottie in the actual application of the table emperor App

Pits that AntDesign A-Tooltip components step on