How does the interesting static model dynamics work?

With the motion capture capability of Huawei 3D Modeling Kit, the human body detection technology, model acceleration and compression technology, and monocular pose estimation algorithm based on deep learning can capture the THREE-DIMENSIONAL information of 24 key points of human bones with only the RGB camera of ordinary mobile phones, easily realizing the dynamic static model. Let’s bring our model to life.

To achieve this functionality is also very simple, take a look at the integration steps!

Application scenarios

The service is widely used in 3D content production, especially in the game, film and television, medical and other industries. For example, the role driving and animation video production in UGC games, the real-time driving of virtual anchors, and rehabilitation guidance in the medical industry.

Integration code

1 Development Preparation

For detailed preparations, see huawei Developer Alliance official website:

Configure AppGallery Connect- Development Readiness -Android-3D Modeling Services (huawei.com)

2 Edit project integration

Before starting API development, you will need to configure AppGallery Connect in 3.3.1. Also, please ensure that your project has configured the Maven repository address of the HMS Core SDK and completed the 3.3.2 integration of the HMS Core SDK for this service.

2.1 Creating a Motion Capture Engine.
// Customize the parameter configuration. Modeling3dMotionCaptureEngineSetting setting = new Modeling3dMotionCaptureEngineSetting. Factory () / / set the test mode. / / Modeling3dMotionCaptureEngineSetting. TYPE_3DSKELETON_QUATERNION: output the body posture corresponding bone quaternions. / / Modeling3dMotionCaptureEngineSetting. TYPE_3DSKELETON: output the body posture corresponding bone point coordinates. .setAnalyzeType(Modeling3dMotionCaptureEngineSetting.TYPE_3DSKELETON_QUATERNION | Modeling3dMotionCaptureEngineSetting.TYPE_3DSKELETON) .create(); Modeling3dMotionCaptureEngine engine = Modeling3dMotionCaptureEngineFactory.getInstance().getMotionCaptureEngine(setting);Copy the code

Modeling3dFrame encapsulates the video frame or still image data from the camera, along with the associated data processing logic.

You can process the video stream yourself and convert the video frame into a Modeling3dFrame object for video frame detection, supporting NV21 format.

Create a Modeling3dFrame object from Android.graphics.bitmap, which is used by the motion capture engine to detect images. Supported image formats include JPG, JPEG, and PNG.

// Create a Modeling3dFrame with a bitmap. Modeling3dFrame frame = Modeling3dFrame.fromBitmap(bitmap); // Create a Modeling3dFrame from the video frame. Modeling3dFrame. Property Property = new Modeling3dFrame. Property. Creator () setFormatType (ImageFormat. NV21) / / set the video frame width. .setwidth (width) // Set the video frame height. .setheight (height) // Sets the rotation Angle of the video frame. .setQuadrant(quadant) // Set video frame number. .setItemIdentity(framIndex) .create(); Modeling3dFrame frame = Modeling3dFrame.fromByteBuffer(byteBuffer, property);Copy the code
2.2 Invoke synchronous or asynchronous methods for motion capture detection.

Example code for asyncAnalyseFrame asynchronous method:

Task<List<Modeling3dMotionCaptureSkeleton>> task = engine.asyncAnalyseFrame(frame); task.addOnSuccessListener(new OnSuccessListener<List<Modeling3dMotionCaptureSkeleton>>() { @Override public void OnSuccess (List < Modeling3dMotionCaptureSkeleton > results) {/ / testing success. }}).addonFailureListener (new OnFailureListener() {@override public void onFailure(Exception e) {// Failed to check. }});Copy the code

AnalyseFrame Synchronization method example code:

SparseArray<Modeling3dMotionCaptureSkeleton> sparseArray = engine.analyseFrame(frame); for (int i = 0; i < sparseArray.size(); I++) {// test result processing. };Copy the code
2.3 After the detection is complete, stop the engine and release detection resources.
try { if (engine ! = null) { engine.stop(); }} catch (IOException e) {// }Copy the code

The Demo presentation

During the integration development process, you can have any questions online bill of lading, there will be someone to answer for you.

For more details >>

Visit the official website of Huawei Developer Alliance to obtain the development guide. The open-source repositories of Huawei mobile services are GitHub and Gitee

Follow us for updates on HMS Core