Since its official launch in 2019, Auteng Persistent Memory has demonstrated extraordinary strength in many fields and won widespread praise. Especially in artificial intelligence, Auteng has been successfully applied in many Internet companies’ AI key businesses.

Auteng persistent memory is how to empower data, accelerate application landing, this time invited Hu Fenghua to write a detailed explanation of persistent memory programming technology.

01 Auteng persistent memory and its usage mode

Intel ® Auton ™ Persistent memory redefines the traditional storage architecture with innovative memory technology, combining cost-effective mass memory with data persistence to provide large persistent memory hierarchies at an affordable price. With groundbreaking performance levels in memory-intensive workloads, virtual machine density, and fast storage capacity, Intel® Optane™PMem accelerates the IT transformation to support the computing demands of the data age.

The all-new PMem 200 series is paired with a third-generation Intel ® XEON ® Expandable processor (Ice Lake, ICX), and the entry-level PMem Series is paired with a second-generation Intel ® Cascade Lake expandable processor (CLX). Software ecosystems built by workloads such as data analytics and infrastructure such as virtualization remain compatible, helping to exploit the potential value of data more effectively. Developers can leverage industry-standard persistent memory programming patterns to build simpler, more powerful applications and ensure that investments in data centers can meet future needs.

Persistent memory products can be used in different ways, some of which correspond to being transparent. For example, all persistent memory products support storage interfaces and standard file apis, just like Solid State disks (SSDS). Or you can configure persistent memory into memory mode, where the system’s persistent memory is used in the same way as system memory. Using persistent memory in these two ways is so straightforward that we don’t need to make any changes to the corresponding memory, and the user doesn’t even feel that they exist except for a significant performance improvement.

But these two methods also have their own weaknesses, for the first method, because basically still use the past software stack, can not give full play to the performance advantages of Auteng persistent memory; As for the second method, although it can obtain a large amount of memory, there is still a gap in access latency performance compared with memory, and it cannot cope with scenarios requiring high memory latency. In addition, it cannot persist data. To take full advantage of Outton persistence performance and persistence, we introduced the AppDirect mode. In this mode, applications can directly access persistent memory from user space in a similar way to memory, which can not only fully exploit the performance advantages of persistent memory, but also carry out in-place persistence of data. This mode requires users to make small changes to the application.

02 Characteristics of persistent memory

The rise of every new technology always leads to new thinking, and persistent memory is no exception. Consider the following characteristics of persistent memory when building and developing solutions:

  • The performance of persistent memory (throughput, latency, and bandwidth) is much higher than that of NAND, but slightly lower than that of DRAM.

  • Unlike NAND, persistent memory is very durable. Their durability is typically orders of magnitude greater than NAND and can outlast the life of the server.

  • Persistent memory modules have much larger capacity than DRAM modules and can share the same memory channels. Applications that support persistent memory can update data in place without serializing/deserializing the data.

  • Persistent memory supports byte addressing (similar to memory). Applications can update only the data they need without incurring any read-modify-write (RMW) overhead.

  • The data is consistent with the CPU cache. Persistent memory provides direct memory access (DMA) and remote Direct memory Access (RDMA) operations.

  • Data written to persistent memory is not lost after a power outage.

  • After the permission check is complete, data on persistent memory can be accessed directly from user space. Data access does not go through any kernel code, file system page cache, or interrupts.

  • Data on persistent memory is immediately available, that is, data is available when the system is powered on. O Applications do not need to spend time warming up the cache. O They provide access to data immediately after memory mapping.

  • Data on persistent memory does not take up DRAM space unless the application copies the data to DRAM for faster access.

  • Data written to the persistent memory module is local to the system. The application is responsible for replicating data between different systems.

Application developers typically consider both memory-resident and storage-resident data structures. In the case of data center applications, developers are careful to maintain consistent data structures in storage, even in the event of a system crash.

This problem can often be solved using logging techniques such as write-ahead logging, where changes are written to the log and then flushed to persistent storage. If the data modification process is interrupted, the application can use the log information to recover the data during the restart. This technique has been around for years; However, proper implementation methods are difficult to develop and time-consuming to maintain. Developers often rely on a combination of databases, programming libraries, and modern file systems to provide consistency.

Ultimately, though, it is up to the application developer to devise a strategy to ensure consistency of data structures in storage both at run time and when recovering from application and system crashes.

03 SNIA NVM programming model

Persistent memory can be accessed directly by applications and performs in-place persistence of data, thus enabling operating systems to support an entirely new programming model that persists data like non-volatile storage devices while providing performance comparable to memory.

Fortunately for developers, while the first generation of persistent memory was still under development, Microsoft Windows and Linux designers, architects and developers have worked with the Storage Network Industry Association (SNIA) to define a SNIA general programming model for persistent memory programming, the SNIA NVM programming model, See Figure 1.

▲ Figure 1 SNIA NVM programming model

Persistent memory is used as block storage

The operating system can detect the presence of persistent memory modules and load device drivers into the I/O subsystem of the operating system, as shown in the left half of Figure 1. The non-volatile dual in-line memory module (NVDIMM) driver has two important functions.

First, it provides a hypervisor interface for system administrators to configure and monitor the state of persistent memory hardware. Second, it has functions similar to storage device drivers. The NVDIMM driver presents persistent memory as a fast block storage device to applications and operating system modules.

This means that applications, file systems, volume managers, and other storage middleware layers can use persistent memory as they currently use storage without modification.

Persistent memory-aware file systems

Another extension to the operating system is support for file system awareness and optimization for persistent memory. Such File systems are known as persistent memory-aware File systems (PMEM-aware File systems).

Current persistent memory-aware file systems include Linux ext4 and XFS, and Microsoft Windows NTFS. As shown in Figure 1, these file systems can either use block drivers in the I/O subsystem, as described above, or they can bypass the I/O subsystem and use persistent memory directly as byte-addressable load/store memory for the fastest and shortest path to access data stored in persistent memory.

In addition to eliminating traditional I/O operations, this path enables small data block writes to perform faster than traditional block storage devices, which require the file system to read the device’s native block size, modify the block, and then write the entire block back to the device. These persistent memory-aware file systems provide familiar standard file apis to applications, including open, close, Read, and write system calls. This allows applications to continue to use familiar file apis while taking advantage of the higher performance of persistent memory.

Persistent Memory Direct Access The persistent memory Direct Access feature (called Direct Access (DAX) in Linux and Windows uses the memory mapped file interface provided by the operating system, but takes full advantage of the native features of persistent memory, which can be used as both data storage and memory. Persistent memory can be mapped to application memory natively, so the operating system does not need to cache files in volatile main memory.

To use DAX, a system administrator needs to create a file system on the persistent memory module and mount the file system in the operating system’s file system tree. For Linux users, persistent memory devices will appear as /dev/pmem* device special files. To display persistent memory physical devices, system administrators can use NDCTL and ipmctl programs.

04 Persistent memory development suite PMDK

As mentioned earlier, applications can access persistent memory directly by using memory-mapped files, but this requires users to modify their applications. To reduce the cost for users to modify applications, we introduced the Persistent Memory Development Suite (PMDK). We’ll show you how to use PMDK to modify applications to efficiently manage byte-addressable data structures that reside in persistent memory.

PMDK library is constructed based on SNIA NVM programming model. They extend the model to varying degrees, some simply encapsulating the operating system-provided primitives into easy-to-use functions, while others provide complex data structures and algorithms for persistent memory. This means that you are responsible for deciding which level of abstraction is best for your use case.

PMDK has evolved over the years and now includes a large number of open source libraries, as shown in Figure 2. These libraries help application developers and system administrators simplify application development and management of persistent memory devices.

PMDK provides two types of libraries:

1) Volatile library: Suitable for users and scenarios that just want to take advantage of large persistent memory capacity.

2) Persistence library: suitable for software that wants to implement fail-safe persistent memory algorithms.

Figure 2 pMDK-related development library

Libmemkind libmemKIND is a user-extensible heap manager built on jemalloc that supports controlling memory features and partitioning the heap between different types of memory. The type of memory is defined by the operating system memory policy applied to the virtual address range. Memory features supported by Memkind in the absence of user extension include control of non-consistent memory access (NUMA) and memory page size features.

The Jemalloc non-standard interface has been extended to support specialized types of requests for virtual memory from the operating system through the Memkind partition interface. With other MemKIND interfaces, you can control and extend memory partitioning features and allocate memory while selecting enabled features. With the memkind interface, you can create and control file-based memory from persistent memory that supports PMEM types.

Libmemcache Libvmemcache is an embeddable lightweight memory cache solution that takes advantage of large storage, such as DAX-enabled persistent memory, with efficient, scalable memory mapping. Libvmemcache has its own unique characteristics:

  • Range-based memory allocators avoid the fragmentation problems that affect most in-memory databases and support caching for extremely high space utilization in most workloads.
  • Buffered least-recently-used (LRU) algorithm combines traditional LRU bidirectional linked lists with non-blocking ring buffers to achieve high scalability on modern multi-core cpus.
  • The Critnib index structure provides high performance while saving a lot of space. The cache is tuned to best handle relatively large values, as small as 256 bytes, but libvmemcache is best suited for expected values larger than 1 KB.

Pmemkv PMEMKV is a general-purpose embedded local key-value store optimized for persistent memory. It is easy to use and comes with many different language integrations, including C, C++, and JavaScript.

The library also has pluggable back-end plug-ins for different storage engines. Although designed primarily for application scenarios that support persistence, it can also be used as a volatile library. Pmemkv was created as a separate project to not only provide cloud native support for a set of libraries in PMDK, but also provide a long-in-built key-value API. One of the main goals of the PMEMkV developers is to create a friendly environment for the open source community to develop a new engine with the help of PMDK and integrate it with other programming languages.

The PMEMKV API is similar to most key-value databases in that it implements multiple storage engines. All of these storage engines have excellent flexibility and functionality. Each engine has different performance characteristics to solve different problems. Therefore, each engine provides different functions, which can be described by the following features:

  • Persistence: A persistence engine ensures that changes can be saved and are power-off safe, while a volatile engine retains its content only for the life of the application.
  • Concurrency: The concurrency engine ensures that certain methods (such as get(), PUT (), remove()) are thread-safe.
  • Sorting of keys: The sorted engine provides range query methods (such as get_above()). pmemkv

It differs from other key-value databases in that it supports direct access to data. This means that reading data from persistent memory does not need to be copied to DRAM.

Direct access to the data can significantly speed up your application. This advantage is most apparent when the program is only interested in part of the data stored in the database. In the traditional approach, you copy all the data into a buffer and then return it to the application. With pMEMKV, we provide a direct pointer to an application that reads only the data it needs.

With its modular design, flexible engine apis, and integration with many common cloud programming languages, PMEMKV has become the first choice for many cloud-native software developers. As an open source lightweight library, it can be easily integrated into existing applications and immediately take advantage of persistent memory; Or develop on this basis for specific application requirements.

Libpmemobj libpmemobj is a C library that provides a store of transaction objects, dynamic memory allocators, transactions, and general functionality for persistent memory programming. The libpmemobj library provides transactional objects stored in persistent memory as direct access (DAX) for applications that require transaction and persistent memory management. The library addresses many of the common algorithmic and data problems encountered in persistent memory programming.

Libpmemobj enables applications to memory-map files on persistent memory-aware file systems, providing direct load/store operations without paging blocks from a block storage device. It bypasses the kernel, avoids context switches and interrupts, and enables applications to read and write directly in byte-addressable persistent memory.

The libpmemobj library provides a convenient API for easily managing the creation and access of memory pools, avoiding the complexities of direct mapping and data synchronization. PMDK also provides the pmemPool program for managing memory pools from the command line. For persistent memory, you can reserve memory for temporary objects using pmemobj_alloc(), pmemobj_reserve(), or pmemobj_ xreserve() in the same way as malloc().

Libpmemobj provides three sets of apis: atomic manipulation API, retention/publication API, and transaction API, which you can use to persist data. Each of these apis provides fail-safe atomicity and consistency to reduce errors when creating applications while ensuring data integrity.

Libpmemobj- CPP The Libpmemobj library provides allocators, transactions, and methods to manipulate objects automatically for low-level system software developers and language creators. However, because it does not modify the compiler, its API is verbose and contains a large number of macros. To simplify persistent memory programming and reduce errors, Intel created a high-level language binding for libpmemobj and added it to PMDK. The C++ binding is libpmemobj-cpp, also known as libpmemobju ++. Libpmemobj is a C++ header-only library that provides a simpler and less error-prone interface for libpmemobj using C++ meta-programming features.

It enables rapid development of persistent memory applications by reusing concepts familiar to C++ programmers, such as smart Pointers and closure transactions, and providing convenient apis to modify structures and classes with only minor changes to functions. The library also comes with stL-compliant custom data structures and containers, so application developers don’t have to redevelop basic algorithms for persistent memory.

Libpmem Libpmem is a low-level C library that provides basic abstractions against operating system rendered primitives. It automatically detects functionality in the platform, selects appropriate persistence semantics and memory transfer (memCPy ()) methods for persistent memory optimization. Most applications need to use at least part of this library.

Libpmem handles persistent memory-related CPU instructions, optimal copying of data to persistent memory, and file mapping.

The libpmem library contains some convenient functions for memory-mapped files. Using these functions to replace memory mapping functions provided by the operating system has the following advantages:

  • Libpmem ensures the correct parameters for the operating system mapping call.
  • Libpmem can detect whether the map is persistent memory and whether it is safe to refresh directly using CPU instructions.
  • Libpmem also provides interfaces to best copy or clear persistent memory areas for persisting data.

Programmers who just want to have completely raw access to persistent memory and don’t need the library to provide allocator or transaction capabilities may want to use libpmem as a basis for development.

For most programmers, libpmem is very low-level and provides maximum programming flexibility, but the resulting coding and debugging costs are relatively high. If not necessary, consider using higher-level libraries to increase development efficiency and reduce debugging costs.

Persistent memory programming resources

Introduction to Intel ® Auteng ™ Persistent Memory: www.intel.cn/content/www… SNIA NVM programming model specifications: www.snia.org/tech_activi… PMDK’s official website: pmem. IO/books: persistent memory programming Chinese version PMDK: github.com/pmem/pmdk libmemkind: memkind. Making. IO/memkind/libvmemcache: Github.com/pmem/vmemca… Pmemkv: github.com/pmem/pmemkv libpmemobj – CPP: github.com/pmem/libpme…

Fenghua Hu is a Cloud Software Architect at Intel’s Upton division. He is committed to exploring the application innovation of persistent Cloud computing, big data, artificial intelligence and Internet of Things. He worked as a software architect and software engineering manager at Alibaba Group and EMC China R&D Center, and has over 15 years of experience in file systems, distributed storage systems, cloud computing and big data.