The article about Qualcomm Camx architecture introduction is still very rare at present, I will try my best to speak as much as possible, speak popular point ~

This article is divided into the following points:

1) Camx overall architecture diagram;

2) Camx basic components and their concepts;

3) Camx code directory structure;

1. Camx overall architecture diagram

At present, the mainstream models of Android, using Qualcomm chips, are basically using CAMX architecture.

The main difference between the previous architecture, called MM-Camera, and the CAMX architecture is that the code for the chip interface layer is migrated from Hardware /qcom to vendor/qcom/proprietary.

Let’s take a look at the overall architecture of CAMX:

From the above figure, we can see that in THE Hal layer, Qualcomm combines its own chip hardware and adds a layer of logic, which is internally named CAMX architecture.

Let’s take a look at the general flow of camera data on Qualcomm CAMX architecture.

  1. From the figure above, we can clearly see that the Camera data comes out from the sensor, first passes through IFE, and then is divided into preview/video and photo taking. If it is preview or video recording, it will first undergo IPE processing and finally output to display. If it is a photo, it is first processed by BSP, then through the JPEG encoder, and finally saved as a picture output.

  2. IFE, IPE, BPS, JPEG, they represent a hardware processing unit inside the chip, data processing in these units is still more complex, in different processing units, there will be some complex algorithm processing, here we first have an understanding, there is a basic concept.

IFE: Image front IPE: Image processing engine BPS: Bayer processing sectionCopy the code

Camx basic components and their concepts

An architecture is always made up of some basic components, so let’s look at the basic components that make up a CAMX architecture.

1) UseCase

A set of streams configured by the client combined with a set of static properties specifying the processing of those Streams (a client-configured set of streams described by a combination of static properties) See createCaptureSession in the Android CameraDevice documentationCopy the code

See  createCaptureSession in the Android CameraDevice

So let’s use this code to understand it better.

If you are familiar with the Camera2 API, you should know that the following code sets up both the preview surface and the recording surface, and then creates a session, which means that I need camera data for both the preview and recording. Suppose I set the preview size to 1080 x 720 and the video is 1080p, then the 1080 x 720 preview +1080p video is a usecase. And so on.

List<Surface> surfaces = new ArrayList<>(); if(previewSurface ! = null && previewSurface.isValid()){ surfaces.add(previewSurface); mPreviewBuilder.addTarget(previewSurface); } if(mMediaRecorder ! = null && mMediaRecorderSurface ! = null && mMediaRecorderSurface.isValid()){ surfaces.add(mMediaRecorderSurface); mPreviewBuilder.addTarget(mMediaRecorderSurface); } mCameraDevice.createCaptureSession(surfaces,... ,...). ;Copy the code

UseCase has many derivative classes in CamX, which is used by CAMX to create different UseCase objects for different streams, manage selected features, and create pipelines and sessions.

2)Feature

Represents a specific function. Features on qualcomm include HDR(high dynamic range), SuperNight (SuperNight), MFNR (multi-frame noise reduction), etc. Usecase selects the corresponding features and then relates a group of pipelines to send request requests from the upper layer. Hal layer will select corresponding feature according to request.

3) Node

A Node is a single abstract module with independent processing functions. It can be a software unit or a hardware unit. Node is a very important parent class in CAMX. It is an intermediate Node for camera requests and is used to process pipeline requests.

Node nodes are crucial in camX Chi architecture, and data processing is done through encapsulated Node nodes.

4) pipeline

A collection of nodes. Pipelines provide a collection of all resources for a single specific function, maintaining all hardware resources and the flow of data.

5) session

A collection of related pipelines that manage abstract control units for pipelines, including at least one pipeline, and control all hardware resources, controlling request flow and data input and output within each pipeline.

6) the Link

Define connection ports (input ports and output ports) for different ports.

7) Port

As the input and output ports of Node, XML files are defined using SrcPort and DstPort structures.

8)Topologies

A topology is A directed Acyclic Graph (DAG) specifying an instantiation of A use case XML definition of use cases and associated topologies For CamX, this graph defines the HW, SW, and non-Qualcomm processing nodes, and the data flow between the nodesCopy the code

Relationships between components

Finally, summarize the relationships between the basic components. According to the requirements, the upper layer configures the corresponding stream, and then selects the corresponding USecase according to the applied stream. After selecting usecase, the required features will be selected, and then different features will be associated with corresponding pipelines. We know that a pipeline consists of a series of nodes, so eventually the stream of the upper config will be handled by each node.

Third, code directory structure

The camX code is in the vendor/qcom/proprietary/ directory, divided into camX and Chi-CDK directories.

1) Directories in CamX

Core is divided into Hal and CHI directories: Hal /, the core implementation module of CamX; Hal /, the Hal/directory that implements hal3 interface; CHI /, the CHI/directory that is responsible for chi's interaction; HWL /, the hardware node with its own algorithm and independent computing capability -- managed by CSL SWL / : Node CSL / : is responsible for the communication module between CAMX and camera driver, and provides a unified camera driver control interface for CAMXCopy the code

2) The directory in chi-CDK

Core / : Houses the core module of CHI implementation, which is responsible for interacting with Camx and implementing CHI's overall framework and business processes. OEM/Qcom/Topolog /: Stores user-defined Usecase XML configuration information. OEM/Qcom /node/: Stores user-defined node functions. OEM/Qcom /module/: Stores configuration files of different sensors. It is needed when initializing the sensor. OEM/QCOM/Tuning /: Stores configuration files for effect parameters in different scenarios. OEM/Qcom /sensor/: Stores only information and register configuration parameters for different sensors. OEM/QCOM/Actuator /: Stores configuration information for different focusing modules. OEM/QCOM /ois/: Stores the configuration information of the anti-shake module. OEM/Qcom /flash/: Stores the configurations of the flash module. OEM/QCOM/EEPROM /: Stores the configuration information of eePROM external storage modules. OEM/Qcom /fd/: stores the configuration information of the face recognition module.Copy the code

* I have been engaged in Android Camera related development for 5 years.

* Currently working in Shenzhen,

* Welcome to follow my wechat official account Xiaochi Notes

* I hope to communicate and learn with more friends

——– 2021.03.27 Shenzhen 23:54

Reference article:

Bbs.huaweicloud.com/blogs/22509…

www.jianshu.com/p/cfb1da9d4…