Have a problem

Recently, I encountered a batch processing task, during which a large amount of data needed to be loaded, including the data to be processed, and some cache data used in the intermediate process. Although you can see how much memory your application is using by monitoring it, you still need to know how much memory is being used by the data and cache loaded in this task, and tune it accordingly.

So how do we know how much of the total memory is due to this batch task? Of course, we can approximate a value based on the number of bytes and the total size of the basic data type, which is a cumbersome and imprecise process.

The basis for problem solving — basic types and how much space they occupy

The data type The number of bytes digits Wrapper class
byte 1 8 Byte
boolean 1 8 Boolean
char 2 16 Character
short 2 16 Short
int 4 32 Integer
float 4 32 Float
double 8 64 Double
long 8 64 Long

1M=1024 bytes the memory size can be calculated based on the number of bytes used.

How do I check the memory usage of running application objects

Jol profile

Jol (JAVA OBJECT LAYOUT) is a tool provided by the OpenJDK to view memory LAYOUT.

  1. Markword specifies the object’s identityHashcode, generation age, lock information, and so on
  2. Classpoint Specifies the class object of the object, a fixed length of 4 bytes.
  3. Basic variables: used to store Java member variables of eight basic types, complete with 4byte steps, and optimize space by memory reordering;
  4. Reference variable: A handle to the reference variable of a class, such as String or Object. Each handle size is 4 bytes. The UseCompressedOops pointer compression parameter is enabled by default in Java8. If this parameter is disabled, it takes 8 bytes.
  5. Complement: The size of the object must be an integer multiple of 8 bytes to complement the number of bytes.
  6. Array length: if it is an array, it takes an additional 4 bytes to store the array length.

Jol’s official website

Jol use

pom.xml

<dependency>
    <groupId>org.openjdk.jol</groupId>
    <artifactId>jol-core</artifactId>
    <version>0.9</version>
</dependency>
Copy the code

Basic type byte

byte b = 'a';
System.out.println(ClassLayout.parseInstance(b).toPrintable());
Copy the code

Output result:

Line 1.2 markword takes up 8 bytes line 3 CLASspoint takes up 4 bytes line 4 byte type takes up 1 byte line 5, since it is not a multiple of 8, completes 3 bytes.

Basic int

int i = 0;
System.out.println(ClassLayout.parseInstance(i).toPrintable());
Copy the code

Output result:

Line 1.2: Markword takes 8 bytes. Line 3: classpoint takes 4 bytes. Line 4: int takes 4 bytes

Base type double

double d = 0;
System.out.println(ClassLayout.parseInstance(d).toPrintable());
Copy the code

Output result:

Line 1.2 Markword takes 8 bytes line 3 classpoint takes 4 bytes line 4 complement line 5 Double takes 8 bytes

An array of Double []

Double[] ds = new Double[10];
System.out.println(ClassLayout.parseInstance(ds).toPrintable());
Copy the code

Output result:

Line 1.2 markword 8 bytes line 3 classpoint 4 bytes line 4 array length 4 bytes line 5?? Why isn’t 8*10=80 bytes? Multiple of 8. I don’t have to fill in

So let’s replace the Double up here with the Double base type

An array of double []

double[] dss = new double[10];
System.out.println(ClassLayout.parseInstance(dss).toPrintable());
Copy the code

Output result:We see that 8 times 10 is 80

From this we infer that if the array is of basic type, yes is the number of bytes taken up by the basic type * array size If the array is of non-basic type, it is the number of bytes taken up by the reference address (4 bytes) * array size

Check it out for yourself

How are reference objects computed

From the above, we can only view the space occupied by objects of some data types. Most object references in real production environments cannot calculate the real space occupied

So let’s go straight to demo

Go straight to code

package top.soft1010.java.base.inst; import java.lang.instrument.Instrumentation; import java.lang.reflect.Array; import java.lang.reflect.Field; import java.lang.reflect.Modifier; import java.util.ArrayDeque; import java.util.Deque; import java.util.HashSet; import java.util.Set; /** * Created by bjzhangjifu on 2021/9/23. */ public class SizeOfObject { static Instrumentation inst; public static void premain(String args, Instrumentation instP) { inst = instP; } /** * directly calculate the current size of the object, including the current class and superclass primitive type instance field size, <br></br> * reference type instance field reference size, instance primitive type array total space occupied, instance reference type array reference size itself occupied space; <br></br> * Public static long <br></br> * * @param obj * @return */ public static long <br></br> * * @param obj * @return */ sizeOf(Object obj) { return inst.getObjectSize(obj); } /** * Recursively calculate the total space occupied by the current object, Includes the instance field size of the current class and superclass and the object size of the instance field reference * * @param objP * @return * @throws IllegalAccessException */ public static Long fullSizeOf(Object objP) throws IllegalAccessException { Set<Object> visited = new HashSet<Object>(); Deque<Object> toBeQueue = new ArrayDeque<>(); toBeQueue.add(objP); long size = 0L; while (toBeQueue.size() > 0) { Object obj = toBeQueue.poll(); // The size of the base type and the length of the reference are taken into account, including the array size += skipObject(visited, obj)? 0L : sizeOf(obj); Class<? > tmpObjClass = obj.getClass(); If (tmPObjclass.isarray ()) {// if (tmPobjclass.getName ().length() > 2) {for (int I = 0, len = Array.getLength(obj); i < len; i++) { Object tmp = Array.get(obj, i); if (tmp ! Add (array.get (obj, I));}}}} else {while (tmpObjClass! = null) { Field[] fields = tmpObjClass.getDeclaredFields(); for (Field field : Fields) {if (Modifier. IsStatic (field. GetModifiers ()) / / static regardless of the | | field. The getType () isPrimitive ()) {/ / basic types do not count the continue repeating;  } field.setAccessible(true); Object fieldValue = field.get(obj); if (fieldValue == null) { continue;  } toBeQueue.add(fieldValue); } tmpObjClass = tmpObjClass.getSuperclass(); } } } return size; } /** * String. Intern objects do not count; The calculated ones don't count, * * @param visited * @param obj * @return */ static Boolean skipObject(Set<Object> visited, Object obj) { if (obj instanceof String && obj == ((String) obj).intern()) { return true;  } return visited.contains(obj); } }Copy the code

Packaged jar

<plugin> <artifactId> Maven-jar-plugin </artifactId> <version>2.4</version> <configuration> <finalName>inst</finalName> <archive> <manifestEntries> <Premain-class>top.soft1010.java.base.inst.SizeOfObject</Premain-class> <Boot-Class-Path></Boot-Class-Path> <Can-Redefine-Classes>false</Can-Redefine-Classes> </manifestEntries> <addMavenDescriptor>false</addMavenDescriptor> </archive> </configuration> </plugin>Copy the code

Execute the command

java -javaagent:.. /.. /target/inst.jar top.soft1010.java.base.inst.TestCopy the code

Basic int

int i = 0;
System.out.println(SizeOfObject.fullSizeOf(i));
Copy the code

The output is 16

Base type double

double j = 0;
System.out.println(SizeOfObject.fullSizeOf(j));
Copy the code

The output is 24

An array of

 double[] ds = new double[10];
 System.out.println(SizeOfObject.fullSizeOf(ds));
Copy the code

Result: 8 bytes markword+4 bytes classpoint+4 bytes array length +double occupied 8 bytes of itself * array length 10=96

 Double[] dds = new Double[10];
 System.out.println(SizeOfObject.fullSizeOf(dds));
Copy the code

Result: 8 bytes markword+4 bytes classpoint+4 bytes array length +Double reference occupancy 4 bytes * array length 10=56

If I assign a value to one of those doubles, what is the result? We found by actual execution that it was 80

Where does 80 come from? Markword (8 bytes) + classpoint (4 bytes) +Double reference (4 bytes) The space taken up by a Double object is 8 bytes markword+4 bytes classpoint+8 bytes Double and 4 complement bits = 24

Introduction of Instrumentation

  • Dynamic Instrumentation using java.lang. Instrument container classes is a new feature in Java SE 5.
  • With Instrumentation, developers can build an application-independent Agent to monitor and assist programs running on the JVM, and even replace and modify the definition of certain classes.
  • This feature provides support for virtual machine monitoring

Instrumentation principles & steps

Developers can have the Instrumentation agent run before main

1. Write premain

public static void premain(String agentArgs,Instrumentation inst);  
public static void premain(String agentArgs);
Copy the code

2. Package jar files

Javaagent :jar file location [= pass premain argument,]