Measure the Real Size of Any Python Object
TL;DR: Use this script to measure sizes of objects in Python. This works for iterables, dicts and subclasses of Python’s built-in Object class.
Edit: Thanks to those who contributed 🙂 Please feel free to submit pull requests for any edge cases you come across.
A few months ago I suspected a global variable to be the root cause of a memory leak. I had enough empirical proof to remove it from our production systems, however not enough to satisfy my inner nerd.
To do that I had to measure the object’s size in memory, which, surprisingly is not a straightforward thing to do in Python. Taking a look at the top StackOverflow answers for “measuring python object sizes” wasn’t satisfying as they only pointed to large chunks of code that I’d have to use to instrument my code and they were way overkill.
What made this worse was the confusing Python docs about sys.getsizeof:
Return the size of an object in bytes. The object can be any type of object. All built-in objects will return correct results, but this does not have to hold true for third-party extensions as it is implementation specific.
Return the size of an object in bytes. The object can be any type of object. All built-in objects will return correct results, but this does not have to hold true for third-party extensions as it is implementation specific.
From an OOP standpoint, this is not true. When you measure a size of an object, you really want the size of all of it’s attributes, and their attributes, etc. sys.getsizeof only gives you the size of the object and their attributes, however it does not recursively add the size of sub-attributes.
So I decided to fill in the gap. I wrote the helper below to recursively measure the size of a Python object (or dict, tuple, list etc). It uses sys.getsizeof internally and it turned out to be pretty easy to write once I figured out enough of the edge cases.
I hope it helps the next time you need to accurately measure the size of an object!
By the way, we’re hiring at Shippo! We work with Python, EmberJS, and PostgreSQL. Our API ships millions of packages around the world.
Shippo is a multi-carrier API and web app that helps retailers, marketplaces and platforms connect to a global network of carriers. Businesses use Shippo to get real-time rates, print labels, automate international paperwork, track packages and facilitate returns. Shippo provides the tools to help businesses succeed through shipping.
