Usually, as a developer, you have to write a program dealing with lots of documents, processing huge volumes of data and information, and keeping the data in the memory. Therefore, this leads to a shortage of memory space that impacts the overall performance of your code.
To prevent the memory shortage, you have to clear the memory by clearing the data or information that is no longer required in the program. This tutorial will guide you on how the memory gets allocated and the different methods to clear the memory in Python.
✨How is Memory Allocated in Python?
In Python, everything is an object. These objects are stored in the memory to be accessed. Before the objects can be stored in memory, a lump of memory initially gets allocated. Python’s memory allocator makes sure that there is available space to store these objects. For this, it interacts with the memory administrator of your operating system.
Python uses dynamic memory allocation. One benefit of this is that it communicates how much memory you need for your program in advance. Another benefit is that data structure manipulation is possible as you can allocate the memory as per your needs if the data structure expands. However, every coin has two sides. Since dynamic memory allocation continues as long as the program is in the state of execution, it consumes more time. Likewise, the memory that has been used should be freed after it has been utilized to overcome this drawback.
✨Garbage Collection in Python
In Python, you cannot explicitly free the memory. To clear memory, you have to ensure that you don’t keep storing the references to the objects. This will ensure that the memory gets cleared when they are garbage-collected. Garbage collection is carried out by a program to clear the previous memory for an object that is not being used.
Since reference cycles take computational work, garbage collection should be automatic. In Python, garbage collection is dependent on object allocations and de-allocations. It saves a lot of issues accordingly that come with manually managing the memory allocation and de-allocation. In particular, Python utilizes reference counting along with garbage collection to clear up the unused memory.
Now, what’s reference counting??
Reference Counting
Whenever the developer creates an object in Python, the underlying object will have both- a reference object and its Python type like lists, dictionary, etc. When the object is referenced in the program, its reference count gets incremented, and it gets decremented when an object is dereferenced. In case an object’s reference count becomes 0, the memory for the object is deallocated.
How to view the reference count?
One can check the reference count of a particular object in Python by importing the sys module. Check the reference count of a variable as shown in the following example:
>>> import sys >>> x = 'hello' >>> sys.getrefcount(x) # 2
Note: The reference count is two as we have first created the variable and then it is passed to the sys.getrefcount() function.
✨Automatic Garbage Collection Using The gc Module
We can also inspect the threshold for new objects (objects in Python known as generation 0 objects) by loading the gc module and requesting for the garbage collection thresholds:
import gc
print("Garbage collection thresholds: " gc.get_threshold())
Garbage collection thresholds: (500, 10, 10)The above system has the default threshold of 500. It implies that when the number of allocations is greater by 500 than the number of de-allocations, then the automatic garbage collector will start running.
Example:
import gc
import pprint
import sys
try:
threshold = int(sys.argv[1])
except (IndexError, ValueError, TypeError):
print 'Invalid threshold'
threshold = 5
class MyObj(object):
def __init__(self, name):
self.name = name
print 'Created', self.name
gc.set_debug(gc.DEBUG_STATS)
gc.set_threshold(threshold, 1, 1)
print 'Thresholds:', gc.get_threshold()
print 'Clearing the collector'
gc.collect()
print
print 'Creating objects'
objs = []
for i in range(10):
objs.append(MyObj(i))Output:
Thresholds: (5, 1, 1) Clearing the collector gc: collecting generation 2... gc: objects in each generation: 218 2683 0 gc: done, 0.0007s elapsed. Creating objects gc: collecting generation 0... gc: objects in each generation: 7 0 2920 gc: done, 0.0000s elapsed. Created 0 Created 1 Created 2 Created 3 Created 4 gc: collecting generation 0... gc: objects in each generation: 6 4 2820 gc: done, 0.0000s elapsed. Created 5 Created 6 Created 7 Created 8 Created 9 gc: collecting generation 2... gc: objects in each generation: 5 6 2817 gc: done, 0.0005s elapsed.
Note: Automatic garbage collection won’t run if your device is running out of memory; rather your application will throw exceptions, which should be taken care of, or your application crashes. In this manner, any part of your code that frees up a huge block of memory has a decent possibility for running manual garbage collection.
✨Manual Garbage Collection Using gc.collect()
The application must be as liberated from reference cycles as possible. Invoking the garbage collector manually during the program execution is the best way to deal with the memory being consumed by reference cycles. To invoke the garbage collector manually you can use the gc.collect().
The gc.collect() method is used to return the number of objects it has collected and de-allocated during the program execution. We can invoke the collector using the following way:
import gc x = gc.collect()
Example:
import gc
import pprint
class Graph(object):
def __init__(self, name):
self.name = name
self.next = None
def set_next(self, next):
print('Linking nodes %s.next = %s' % (self, next))
self.next = next
def __repr__(self):
return '%s(%s)' % (self.__class__.__name__, self.name)
# Construct a graph cycle
one = Graph('one')
two = Graph('two')
three = Graph('three')
one.set_next(two)
two.set_next(three)
three.set_next(one)
print()
# Remove references to the graph nodes in this module's namespace
one = two = three = None
# Show the effect of garbage collection
for i in range(2):
print('Collecting %d ...' % i)
n = gc.collect()
print('Unreachable objects:', n)
print('Remaining Garbage:', end=' ')
pprint.pprint(gc.garbage)
print()Output:
Linking nodes Graph(one).next = Graph(two) Linking nodes Graph(two).next = Graph(three) Linking nodes Graph(three).next = Graph(one) Collecting 0 ... Unreachable objects: 6 Remaining Garbage: [] Collecting 1 ... Unreachable objects: 0 Remaining Garbage: []
In the above program, the cycle is cleared when garbage collection runs the initial time since nothing refers to the Graph nodes aside from themselves. The gc.collect() method returns the number of inaccessible objects found.
Please stay tuned and subscribe for more solutions and interesting discussions in the future. Happy Learning!
Where to Go From Here?
Enough theory. Let’s get some practice!
Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.
To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?
You build high-value coding skills by working on practical coding projects!
Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?
🚀 If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.
If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.