<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Memory Management and Limits — Python Module of the Week</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '1.132',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
<link rel="top" title="Python Module of the Week" href="../index.html" />
<link rel="up" title="sys – System-specific Configuration" href="index.html" />
<link rel="next" title="Exception Handling" href="exceptions.html" />
<link rel="prev" title="Runtime Environment" href="runtime.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="exceptions.html" title="Exception Handling"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="runtime.html" title="Runtime Environment"
accesskey="P">previous</a> |</li>
<li><a href="../contents.html">PyMOTW</a> »</li>
<li><a href="../runtime_services.html" >Python Runtime Services</a> »</li>
<li><a href="index.html" accesskey="U">sys – System-specific Configuration</a> »</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="../contents.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Memory Management and Limits</a><ul>
<li><a class="reference internal" href="#reference-counts">Reference Counts</a></li>
<li><a class="reference internal" href="#object-size">Object Size</a></li>
<li><a class="reference internal" href="#recursion">Recursion</a></li>
<li><a class="reference internal" href="#maximum-values">Maximum Values</a></li>
<li><a class="reference internal" href="#floating-point-values">Floating Point Values</a></li>
<li><a class="reference internal" href="#byte-ordering">Byte Ordering</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="runtime.html"
title="previous chapter">Runtime Environment</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="exceptions.html"
title="next chapter">Exception Handling</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/sys/limits.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="memory-management-and-limits">
<span id="sys-limits"></span><h1>Memory Management and Limits<a class="headerlink" href="#memory-management-and-limits" title="Permalink to this headline">¶</a></h1>
<p><a class="reference internal" href="index.html#module-sys" title="sys: System-specific configuration"><tt class="xref py py-mod docutils literal"><span class="pre">sys</span></tt></a> includes several functions for understanding and
controlling memory usage.</p>
<div class="section" id="reference-counts">
<h2>Reference Counts<a class="headerlink" href="#reference-counts" title="Permalink to this headline">¶</a></h2>
<p>Python uses <em>reference counting</em> and <em>garbage collection</em> for
automatic memory management. An object is automatically marked to be
collected when its reference count drops to zero. To examine the
reference count of an existing object, use <tt class="xref py py-func docutils literal"><span class="pre">getrefcount()</span></tt>.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">sys</span>
<span class="n">one</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">print</span> <span class="s">'At start :'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">getrefcount</span><span class="p">(</span><span class="n">one</span><span class="p">)</span>
<span class="n">two</span> <span class="o">=</span> <span class="n">one</span>
<span class="k">print</span> <span class="s">'Second reference :'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">getrefcount</span><span class="p">(</span><span class="n">one</span><span class="p">)</span>
<span class="k">del</span> <span class="n">two</span>
<span class="k">print</span> <span class="s">'After del :'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">getrefcount</span><span class="p">(</span><span class="n">one</span><span class="p">)</span>
</pre></div>
</div>
<p>The count is actually one higher than expected because there is a
temporary reference to the object held by <tt class="xref py py-func docutils literal"><span class="pre">getrefcount()</span></tt> itself.</p>
<div class="highlight-python"><pre>$ python sys_getrefcount.py
At start : 2
Second reference : 3
After del : 2</pre>
</div>
<div class="admonition-see-also admonition seealso">
<p class="first admonition-title">See also</p>
<dl class="last docutils">
<dt><a class="reference internal" href="../gc/index.html#module-gc" title="gc: Garbage Collector"><tt class="xref py py-mod docutils literal"><span class="pre">gc</span></tt></a></dt>
<dd>Control the garbage collector via the functions exposed in <a class="reference internal" href="../gc/index.html#module-gc" title="gc: Garbage Collector"><tt class="xref py py-mod docutils literal"><span class="pre">gc</span></tt></a>.</dd>
</dl>
</div>
</div>
<div class="section" id="object-size">
<h2>Object Size<a class="headerlink" href="#object-size" title="Permalink to this headline">¶</a></h2>
<p>Knowing how many references an object has may help find cycles or a
memory leak, but it isn’t enough to determine what objects are
consuming the <em>most</em> memory. That requires knowledge about how big
objects are.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">class</span> <span class="nc">OldStyle</span><span class="p">:</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">NewStyle</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">for</span> <span class="n">obj</span> <span class="ow">in</span> <span class="p">[</span> <span class="p">[],</span> <span class="p">(),</span> <span class="p">{},</span> <span class="s">'c'</span><span class="p">,</span> <span class="s">'string'</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mf">2.3</span><span class="p">,</span>
<span class="n">OldStyle</span><span class="p">,</span> <span class="n">OldStyle</span><span class="p">(),</span> <span class="n">NewStyle</span><span class="p">,</span> <span class="n">NewStyle</span><span class="p">(),</span>
<span class="p">]:</span>
<span class="k">print</span> <span class="s">'</span><span class="si">%10s</span><span class="s"> : </span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">getsizeof</span><span class="p">(</span><span class="n">obj</span><span class="p">))</span>
</pre></div>
</div>
<p><tt class="xref py py-func docutils literal"><span class="pre">getsizeof()</span></tt> reports the size in bytes.</p>
<div class="highlight-python"><pre>$ python sys_getsizeof.py
list : 72
tuple : 56
dict : 280
str : 38
str : 43
int : 24
float : 24
classobj : 104
instance : 72
type : 904
NewStyle : 64</pre>
</div>
<p>The reported size for a custom class does not include the size of the
attribute values.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">class</span> <span class="nc">WithoutAttributes</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">WithAttributes</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">a</span> <span class="o">=</span> <span class="s">'a'</span>
<span class="bp">self</span><span class="o">.</span><span class="n">b</span> <span class="o">=</span> <span class="s">'b'</span>
<span class="k">return</span>
<span class="n">without_attrs</span> <span class="o">=</span> <span class="n">WithoutAttributes</span><span class="p">()</span>
<span class="k">print</span> <span class="s">'WithoutAttributes:'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">getsizeof</span><span class="p">(</span><span class="n">without_attrs</span><span class="p">)</span>
<span class="n">with_attrs</span> <span class="o">=</span> <span class="n">WithAttributes</span><span class="p">()</span>
<span class="k">print</span> <span class="s">'WithAttributes:'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">getsizeof</span><span class="p">(</span><span class="n">with_attrs</span><span class="p">)</span>
</pre></div>
</div>
<p>This can give a false impression of the amount of memory being
consumed.</p>
<div class="highlight-python"><pre>$ python sys_getsizeof_object.py
WithoutAttributes: 64
WithAttributes: 64</pre>
</div>
<p>For a more complete estimate of the space used by a class, provide a
<tt class="xref py py-func docutils literal"><span class="pre">__sizeof__()</span></tt> method to compute the value by aggregating the
sizes of attributes of an object.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">class</span> <span class="nc">WithAttributes</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">a</span> <span class="o">=</span> <span class="s">'a'</span>
<span class="bp">self</span><span class="o">.</span><span class="n">b</span> <span class="o">=</span> <span class="s">'b'</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">__sizeof__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">object</span><span class="o">.</span><span class="n">__sizeof__</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">+</span> \
<span class="nb">sum</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">getsizeof</span><span class="p">(</span><span class="n">v</span><span class="p">)</span> <span class="k">for</span> <span class="n">v</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">__dict__</span><span class="o">.</span><span class="n">values</span><span class="p">())</span>
<span class="n">my_inst</span> <span class="o">=</span> <span class="n">WithAttributes</span><span class="p">()</span>
<span class="k">print</span> <span class="n">sys</span><span class="o">.</span><span class="n">getsizeof</span><span class="p">(</span><span class="n">my_inst</span><span class="p">)</span>
</pre></div>
</div>
<p>This version adds the base size of the object to the sizes of all of
the attributes stored in the internal <tt class="xref py py-data docutils literal"><span class="pre">__dict__</span></tt>.</p>
<div class="highlight-python"><pre>$ python sys_getsizeof_custom.py
140</pre>
</div>
</div>
<div class="section" id="recursion">
<h2>Recursion<a class="headerlink" href="#recursion" title="Permalink to this headline">¶</a></h2>
<p>Allowing infinite recursion in a Python application may introduce a
stack overflow in the interpreter itself, leading to a crash. To
eliminate this situation, the interpreter provides a way to control
the maximum recursion depth using <tt class="xref py py-func docutils literal"><span class="pre">setrecursionlimit()</span></tt> and
<tt class="xref py py-func docutils literal"><span class="pre">getrecursionlimit()</span></tt>.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">print</span> <span class="s">'Initial limit:'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">getrecursionlimit</span><span class="p">()</span>
<span class="n">sys</span><span class="o">.</span><span class="n">setrecursionlimit</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'Modified limit:'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">getrecursionlimit</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">generate_recursion_error</span><span class="p">(</span><span class="n">i</span><span class="p">):</span>
<span class="k">print</span> <span class="s">'generate_recursion_error(</span><span class="si">%s</span><span class="s">)'</span> <span class="o">%</span> <span class="n">i</span>
<span class="n">generate_recursion_error</span><span class="p">(</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">generate_recursion_error</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">RuntimeError</span><span class="p">,</span> <span class="n">err</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'Caught exception:'</span><span class="p">,</span> <span class="n">err</span>
</pre></div>
</div>
<p>Once the recursion limit is reached, the interpreter raises a
<a class="reference internal" href="../exceptions/index.html#exceptions-runtimeerror"><em>RuntimeError</em></a> exception so the
program has an opportunity to handle the situation.</p>
<div class="highlight-python"><pre>$ python sys_recursionlimit.py
Initial limit: 1000
Modified limit: 10
generate_recursion_error(1)
generate_recursion_error(2)
generate_recursion_error(3)
generate_recursion_error(4)
generate_recursion_error(5)
generate_recursion_error(6)
generate_recursion_error(7)
generate_recursion_error(8)
Caught exception: maximum recursion depth exceeded while getting the str of an object</pre>
</div>
</div>
<div class="section" id="maximum-values">
<h2>Maximum Values<a class="headerlink" href="#maximum-values" title="Permalink to this headline">¶</a></h2>
<p>Along with the runtime configurable values, <a class="reference internal" href="index.html#module-sys" title="sys: System-specific configuration"><tt class="xref py py-mod docutils literal"><span class="pre">sys</span></tt></a> includes
variables defining the maximum values for types that vary from system
to system.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">print</span> <span class="s">'maxint :'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">maxint</span>
<span class="k">print</span> <span class="s">'maxsize :'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">maxsize</span>
<span class="k">print</span> <span class="s">'maxunicode:'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">maxunicode</span>
</pre></div>
</div>
<p><tt class="xref py py-const docutils literal"><span class="pre">maxint</span></tt> is the largest representable regular integer.
<tt class="xref py py-const docutils literal"><span class="pre">maxsize</span></tt> is the maximum size of a list, dictionary, string, or
other data structure dictated by the C interpreter’s size type.
<tt class="xref py py-const docutils literal"><span class="pre">maxunicode</span></tt> is the largest integer Unicode point supported by
the interpreter as currently configured.</p>
<div class="highlight-python"><pre>$ python sys_maximums.py
maxint : 9223372036854775807
maxsize : 9223372036854775807
maxunicode: 65535</pre>
</div>
</div>
<div class="section" id="floating-point-values">
<h2>Floating Point Values<a class="headerlink" href="#floating-point-values" title="Permalink to this headline">¶</a></h2>
<p>The structure <tt class="xref py py-data docutils literal"><span class="pre">float_info</span></tt> contains information about the
floating point type representation used by the interpreter, based on
the underlying system’s float implementation.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">print</span> <span class="s">'Smallest difference (epsilon):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">epsilon</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'Digits (dig) :'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">dig</span>
<span class="k">print</span> <span class="s">'Mantissa digits (mant_dig):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">mant_dig</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'Maximum (max):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">max</span>
<span class="k">print</span> <span class="s">'Minimum (min):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">min</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'Radix of exponents (radix):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">radix</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'Maximum exponent for radix (max_exp):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">max_exp</span>
<span class="k">print</span> <span class="s">'Minimum exponent for radix (min_exp):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">min_exp</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'Maximum exponent for power of 10 (max_10_exp):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">max_10_exp</span>
<span class="k">print</span> <span class="s">'Minimum exponent for power of 10 (min_10_exp):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">min_10_exp</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'Rounding for addition (rounds):'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">float_info</span><span class="o">.</span><span class="n">rounds</span>
</pre></div>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">These values depend on the compiler and underlying system. These
examples were produced on OS X 10.6.4.</p>
</div>
<div class="highlight-python"><pre>$ python sys_float_info.py
Smallest difference (epsilon): 2.22044604925e-16
Digits (dig) : 15
Mantissa digits (mant_dig): 53
Maximum (max): 1.79769313486e+308
Minimum (min): 2.22507385851e-308
Radix of exponents (radix): 2
Maximum exponent for radix (max_exp): 1024
Minimum exponent for radix (min_exp): -1021
Maximum exponent for power of 10 (max_10_exp): 308
Minimum exponent for power of 10 (min_10_exp): -307
Rounding for addition (rounds): 1</pre>
</div>
<div class="admonition-see-also admonition seealso">
<p class="first admonition-title">See also</p>
<p class="last">The <tt class="docutils literal"><span class="pre">float.h</span></tt> C header file for the local compiler contains more
details about these settings.</p>
</div>
</div>
<div class="section" id="byte-ordering">
<h2>Byte Ordering<a class="headerlink" href="#byte-ordering" title="Permalink to this headline">¶</a></h2>
<p><tt class="xref py py-const docutils literal"><span class="pre">byteorder</span></tt> is set to the native byte order.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">sys</span>
<span class="k">print</span> <span class="n">sys</span><span class="o">.</span><span class="n">byteorder</span>
</pre></div>
</div>
<p>The value is either <tt class="docutils literal"><span class="pre">big</span></tt> for big-endian or <tt class="docutils literal"><span class="pre">little</span></tt> for
little-endian.</p>
<div class="highlight-python"><pre>$ python sys_byteorder.py
little</pre>
</div>
<div class="admonition-see-also admonition seealso">
<p class="first admonition-title">See also</p>
<dl class="last docutils">
<dt><a class="reference external" href="http://en.wikipedia.org/wiki/Byte_order">Wikipedia: Endianness</a></dt>
<dd>Description of big and little endian memory systems.</dd>
<dt><a class="reference internal" href="../array/index.html#module-array" title="array: Manage sequences of fixed-type numerical data efficiently."><tt class="xref py py-mod docutils literal"><span class="pre">array</span></tt></a>, <a class="reference internal" href="../struct/index.html#module-struct" title="struct: Convert between strings and binary data."><tt class="xref py py-mod docutils literal"><span class="pre">struct</span></tt></a></dt>
<dd>Other modules that depend on the byte order.</dd>
</dl>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="exceptions.html" title="Exception Handling"
>next</a> |</li>
<li class="right" >
<a href="runtime.html" title="Runtime Environment"
>previous</a> |</li>
<li><a href="../contents.html">PyMOTW</a> »</li>
<li><a href="../runtime_services.html" >Python Runtime Services</a> »</li>
<li><a href="index.html" >sys – System-specific Configuration</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright Doug Hellmann.
Last updated on Oct 24, 2010.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
<br/><a href="http://creativecommons.org/licenses/by-nc-sa/3.0/us/" rel="license"><img alt="Creative Commons License" style="border-width:0" src="http://i.creativecommons.org/l/by-nc-sa/3.0/us/88x31.png"/></a>
</div>
</body>
</html>