preface

In the previous article, Garbage collection 1, we talked about Garbage collection(Ruby and Python). In this section, we will continue to cover some of the applications and considerations of the GC module

First, garbage recycling mechanism

Garbage collection in Python is dominated by reference counting and supplemented by generational collection.

1. The case that causes the reference count to +1

  • Object is created, for example a=23
  • Object is referenced, for example b=a
  • Objects are passed as arguments to a function, such as func(a).
  • The object is stored as an element in a container, for example list1=[a,a]

2. The condition that causes the reference count to be -1

  • The alias of an object is explicitly destroyed, for example del a
  • The alias of the object is given to the new object, for example a=24
  • An object leaves its scope, such as a local variable in a func function at the end of f (global variables do not)
  • The container in which the object resides is destroyed, or the object is deleted from the container

3. View the reference count of an object

In [1]: import sys

In [2]: a = "hello world"

In [3]: sys.getrefcount(a)
Out[3]: 2
Copy the code

You can look at the reference count of object A, but it is 1 higher than the normal count because a is passed in when the function is called, which increases the reference count of object A by 1

Second, circular references cause memory leaks

The flaw in reference counting is the problem of circular references

import sys a = "hello world" sys.getrefcount(a) import gc class ClassA(): def __init__(self): print('object born,id:%s'%str(hex(id(self)))) def f2(): while True: C1 = ClassA() c2 = ClassA() c1.t = c2 c2.t = c1 del c1 del c2Copy the code

If you run f2(), the memory footprint of the process increases.

  • C1. t=c2 and c2.t=c1.
  • After del c1, the reference count of the object in memory 1 becomes 1. Since it is not 0, the object in memory 1 will not be destroyed, so the reference count of the object in memory 2 will still be 2. After del c2, the reference count of the object in memory 1 and the reference count of the object in memory 2 will be 1.
  • Although both objects are destructible, the garbage collector will not collect them due to circular references, resulting in a memory leak.

Three, garbage recycling

import gc class ClassA(): def __init__(self): print('object born,id:%s'%str(hex(id(self)))) #def __del__(self): # print('object del,id:%s'%str(hex(id(self)))) def f3(): print("-----0------") #print(gc.collect()) c1 = ClassA() c2 = ClassA() c1.t = c2 c2.t = c1 print("-----1------") del c1 Del c2 print (" -- -- -- -- -- 2 -- -- -- -- -- - ") print (gc. Garbage) print (" -- -- -- -- -- 3 -- -- -- -- -- - ") print (gc) collect ()) # explicit garbage recycling print (" -- -- -- -- -- 4 -- -- -- -- -- - ") Print (gc. Garbage) print (" -- -- -- -- -- -- -- -- -- -- - "5) if __name__ = =" __main__ ': gc. Set_debug (gc. DEBUG_LEAK) # set gc module log f3 ()Copy the code

Python3 results are as follows:

-----0------
object born,id:0x7fcd059190f0
object born,id:0x7fcd05919240
-----1------
-----2------
[]
-----3------
gc: collectable <ClassA 0x7fcd059190f0>
gc: collectable <ClassA 0x7fcd05919240>
gc: collectable <dict 0x7fcd05989d48>
gc: collectable <dict 0x7fcd058f24c8>
4
-----4------
[<__main__.ClassA object at 0x7fcd059190f0>, <__main__.ClassA object at 0x7fcd05919240>, {'t': <__main__.ClassA object at 0x7fcd05919240>}, {'t': <__main__.ClassA object at 0x7fcd059190f0>}]
-----5------
gc: collectable <module 0x7fcd059715e8>
gc: collectable <dict 0x7fcd0597af08>
gc: collectable <builtin_function_or_method 0x7fcd0596fdc8>
...
Copy the code

Description:

  • The collected objects are placed in the gC.garbage list
  • Gc.collect () returns the number of unreachable objects, with 4 equal to two objects and their dict equivalents

Garbage collection can be triggered in three ways:

  1. Call gc. Collect (),
  2. When the gc module’s counter reaches its threshold.
  3. When the program exits

Iv. Analysis of common functions of GC module

The GC module provides an interface for developers to set garbage collection options. As mentioned above, one of the drawbacks of the reference counting approach to memory management is circular references, and one of the main functions of the GC module is to solve the problem of circular references.

Common functions:

Gc.set_debug (flags) specifies the gc debug log. Gc.set_debug (flags) specifies the GC debug log. Gc.set_debug (flags) specifies the GC debug log. If no arguments are passed, execute a full collection, which is equal to pass 2. 3. The frequency of automatic garbage collection in the GC module obtained by gc.get_threshold(). Gc.set_threshold (threshold0[, threshold1[, threshold2]) sets the frequency at which an automatic garbage collection is performed.

Automatic garbage collection mechanism for GC module

You must import the GC module and is_enable()=True to start automatic garbage collection. The main purpose of this mechanism is to find and dispose of unreachable garbage objects. Garbage collection = garbage inspection + garbage collection In Python, generational collection is used. The object is divided into three generations. At the beginning, the object is placed in the first generation when it is created. If the object survives the garbage check of the first generation, it will be placed in the second generation. The GC module contains a list of counters of length 3, which can be obtained by gC.get_count (). For example, (488,3,0), where 488 refers to the number of memory allocated by Python minus the number of memory freed since the last generation garbage check. Note the memory allocation, not the increase in the reference count. Such as:

Print (gc) get_count ()) # 590,8,0) (a = ClassA () print (gc) get_count ()) # (590,8,0) del a print (gc) get_count ()) # (590,8,0)Copy the code

3 refers to the number of second-generation garbage checks since the last second-generation garbage checks. Similarly, 0 refers to the number of third-generation garbage checks since the last third-generation garbage checks. The GC module has an automatic garbage collection threshold, that is, a tuple of length 3 obtained through the gC. get_threshold function. For example, each time the counter is increased, the GC module checks whether the increased count has reached the threshold number. If so, the corresponding algebraic garbage check is performed. Then reset the counter. For example, suppose the threshold is (700,10,10) :

Collect (0) when the counter increases from (699,3,0) to (700,3,0), the gc module will perform gc.collect(0), which checks for garbage of generation objects and resets the counter when the counter increases from (699,9,0) to (700,9,0), Collect (1) when the counter increases from (699,9,9) to (700,9,9), the gc module performs gc.collect(2), which checks the garbage of the first, second, and third objects and resets the counterCopy the code

Pay attention to the point

The only thing the GC module can’t handle is that all classes referenced in a loop have __del__ methods, so avoid defining __del__ methods in projects

import gc
class ClassA():
    pass
    #def __del__(self):
    #    print('object born,id:%s'%str(hex(id(self))))

gc.set_debug(gc.DEBUG_LEAK)
a = ClassA()
b = ClassA()
a.next = b
b.prev = a
print("--1--")
print(gc.collect())
print("--2--")
del a
print("--3--")
del b
print("--3-1--")
print(gc.collect())
print("--4--")
Copy the code

The running results are as follows:

--1--
0
--2--
--3--
--3-1--
gc: collectable <ClassA 0x7f599dc690f0>
gc: collectable <ClassA 0x7f599dc69160>
gc: collectable <dict 0x7f599dcdcd48>
gc: collectable <dict 0x7f599dcdcdc8>
4
--4--
gc: collectable <module 0x7f599dcc45e8>
gc: collectable <dict 0x7f599dccdf08>
gc: collectable <builtin_function_or_method 0x7f599dcc2dc8>
...
Copy the code

If del is turned on, the result is:

--1--
0
--2--
--3--
--3-1--
gc: collectable <ClassA 0x7fb236853128>
gc: collectable <ClassA 0x7fb236853160>
gc: collectable <dict 0x7fb2368c5d48>
gc: collectable <dict 0x7fb2368c5ec8>
object born,id:0x7fb236853128
object born,id:0x7fb236853160
4
--4--
gc: collectable <module 0x7fb2368ad5e8>
gc: collectable <dict 0x7fb2368b6f08>
gc: collectable <builtin_function_or_method 0x7fb2368abdc8>
...
Copy the code