<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>pickle and cPickle – Python object serialization — Python Module of the Week</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '1.132',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
<link rel="top" title="Python Module of the Week" href="../index.html" />
<link rel="up" title="Data Persistence" href="../persistence.html" />
<link rel="next" title="shelve – Persistent storage of arbitrary Python objects" href="../shelve/index.html" />
<link rel="prev" title="gdbm – GNU’s version of the dbm library" href="../gdbm/index.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="../shelve/index.html" title="shelve – Persistent storage of arbitrary Python objects"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="../gdbm/index.html" title="gdbm – GNU’s version of the dbm library"
accesskey="P">previous</a> |</li>
<li><a href="../contents.html">PyMOTW</a> »</li>
<li><a href="../persistence.html" accesskey="U">Data Persistence</a> »</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="../contents.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">pickle and cPickle – Python object serialization</a><ul>
<li><a class="reference internal" href="#importing">Importing</a></li>
<li><a class="reference internal" href="#encoding-and-decoding-data-in-strings">Encoding and Decoding Data in Strings</a></li>
<li><a class="reference internal" href="#working-with-streams">Working with Streams</a></li>
<li><a class="reference internal" href="#problems-reconstructing-objects">Problems Reconstructing Objects</a></li>
<li><a class="reference internal" href="#circular-references">Circular References</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="../gdbm/index.html"
title="previous chapter">gdbm – GNU’s version of the dbm library</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="../shelve/index.html"
title="next chapter">shelve – Persistent storage of arbitrary Python objects</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/pickle/index.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="module-pickle">
<span id="pickle-and-cpickle-python-object-serialization"></span><h1>pickle and cPickle – Python object serialization<a class="headerlink" href="#module-pickle" title="Permalink to this headline">¶</a></h1>
<span class="target" id="module-cPickle"></span><table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field"><th class="field-name">Purpose:</th><td class="field-body">Python object serialization</td>
</tr>
<tr class="field"><th class="field-name">Python Version:</th><td class="field-body">pickle at least 1.4, cPickle 1.5</td>
</tr>
</tbody>
</table>
<p>The <a class="reference internal" href="#module-pickle" title="pickle: Python object serialization"><tt class="xref py py-mod docutils literal"><span class="pre">pickle</span></tt></a> module implements an algorithm for turning an
arbitrary Python object into a series of bytes. This process is also
called <em>serializing</em>” the object. The byte stream representing the
object can then be transmitted or stored, and later reconstructed to
create a new object with the same characteristics.</p>
<p>The <a class="reference internal" href="#module-cPickle" title="cPickle: Python object serialization"><tt class="xref py py-mod docutils literal"><span class="pre">cPickle</span></tt></a> module implements the same algorithm, in C instead
of Python. It is many times faster than the Python implementation, but
does not allow the user to subclass from Pickle. If subclassing is not
important for your use, you probably want to use cPickle.</p>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p class="last">The documentation for pickle makes clear that it offers no security
guarantees. Be careful if you use pickle for inter-process communication or
data storage. Do not trust data you cannot verify as secure.</p>
</div>
<div class="section" id="importing">
<h2>Importing<a class="headerlink" href="#importing" title="Permalink to this headline">¶</a></h2>
<p>It is common to first try to import cPickle, giving an alias of
“pickle”. If that import fails for any reason, you can then fall back
on the native Python implementation in the pickle module. This gives
you the faster implementation, if it is available, and the portable
implementation otherwise.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span>
<span class="k">except</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">pickle</span>
</pre></div>
</div>
</div>
<div class="section" id="encoding-and-decoding-data-in-strings">
<h2>Encoding and Decoding Data in Strings<a class="headerlink" href="#encoding-and-decoding-data-in-strings" title="Permalink to this headline">¶</a></h2>
<p>This first example encodes a data structure as a string, then prints
the string to the console. It uses a data structure made up of
entirely native types. Instances of any class can be pickled, as will
be illustrated in a later example. Use <tt class="docutils literal"><span class="pre">pickle.dumps()</span></tt> to create a
string representation of the value of the object.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span>
<span class="k">except</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">import</span> <span class="nn">pprint</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="k">print</span> <span class="s">'DATA:'</span><span class="p">,</span>
<span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="n">data_string</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'PICKLE:'</span><span class="p">,</span> <span class="n">data_string</span>
</pre></div>
</div>
<p>By default, the pickle will contain only ASCII characters. A more
efficient binary format is also available, but all of the examples
here use the ASCII output because it is easier to understand in print.</p>
<div class="highlight-python"><pre>$ python pickle_string.py
DATA:[{'a': 'A', 'b': 2, 'c': 3.0}]
PICKLE: (lp1
(dp2
S'a'
S'A'
sS'c'
F3
sS'b'
I2
sa.</pre>
</div>
<p>Once the data is serialized, you can write it to a file, socket, pipe, etc.
Then later you can read the file and unpickle the data to construct a new
object with the same values.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span>
<span class="k">except</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">import</span> <span class="nn">pprint</span>
<span class="n">data1</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="k">print</span> <span class="s">'BEFORE:'</span><span class="p">,</span>
<span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">data1</span><span class="p">)</span>
<span class="n">data1_string</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data1</span><span class="p">)</span>
<span class="n">data2</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">data1_string</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'AFTER:'</span><span class="p">,</span>
<span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">data2</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'SAME?:'</span><span class="p">,</span> <span class="p">(</span><span class="n">data1</span> <span class="ow">is</span> <span class="n">data2</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'EQUAL?:'</span><span class="p">,</span> <span class="p">(</span><span class="n">data1</span> <span class="o">==</span> <span class="n">data2</span><span class="p">)</span>
</pre></div>
</div>
<p>As you see, the newly constructed object is the equal to but not the same
object as the original. No surprise there.</p>
<div class="highlight-python"><pre>$ python pickle_unpickle.py
BEFORE:[{'a': 'A', 'b': 2, 'c': 3.0}]
AFTER:[{'a': 'A', 'b': 2, 'c': 3.0}]
SAME?: False
EQUAL?: True</pre>
</div>
</div>
<div class="section" id="working-with-streams">
<h2>Working with Streams<a class="headerlink" href="#working-with-streams" title="Permalink to this headline">¶</a></h2>
<p>In addition to <tt class="docutils literal"><span class="pre">dumps()</span></tt> and <tt class="docutils literal"><span class="pre">loads()</span></tt>, pickle provides a couple
of convenience functions for working with file-like streams. It is
possible to write multiple objects to a stream, and then read them
from the stream without knowing in advance how many objects are
written or how big they are.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span>
<span class="k">except</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">import</span> <span class="nn">pprint</span>
<span class="kn">from</span> <span class="nn">StringIO</span> <span class="kn">import</span> <span class="n">StringIO</span>
<span class="k">class</span> <span class="nc">SimpleObject</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="n">l</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="n">l</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name_backwards</span> <span class="o">=</span> <span class="s">''</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">l</span><span class="p">)</span>
<span class="k">return</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">data</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">SimpleObject</span><span class="p">(</span><span class="s">'pickle'</span><span class="p">))</span>
<span class="n">data</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">SimpleObject</span><span class="p">(</span><span class="s">'cPickle'</span><span class="p">))</span>
<span class="n">data</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">SimpleObject</span><span class="p">(</span><span class="s">'last'</span><span class="p">))</span>
<span class="c"># Simulate a file with StringIO</span>
<span class="n">out_s</span> <span class="o">=</span> <span class="n">StringIO</span><span class="p">()</span>
<span class="c"># Write to the stream</span>
<span class="k">for</span> <span class="n">o</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'WRITING: </span><span class="si">%s</span><span class="s"> (</span><span class="si">%s</span><span class="s">)'</span> <span class="o">%</span> <span class="p">(</span><span class="n">o</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">o</span><span class="o">.</span><span class="n">name_backwards</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">o</span><span class="p">,</span> <span class="n">out_s</span><span class="p">)</span>
<span class="n">out_s</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="c"># Set up a read-able stream</span>
<span class="n">in_s</span> <span class="o">=</span> <span class="n">StringIO</span><span class="p">(</span><span class="n">out_s</span><span class="o">.</span><span class="n">getvalue</span><span class="p">())</span>
<span class="c"># Read the data</span>
<span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">o</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">in_s</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">EOFError</span><span class="p">:</span>
<span class="k">break</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'READ: </span><span class="si">%s</span><span class="s"> (</span><span class="si">%s</span><span class="s">)'</span> <span class="o">%</span> <span class="p">(</span><span class="n">o</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">o</span><span class="o">.</span><span class="n">name_backwards</span><span class="p">)</span>
</pre></div>
</div>
<p>The example simulates streams using StringIO buffers, so we have to
play a little trickery to establish the readable stream. A simple
database format could use pickles to store objects, too, though
<a class="reference internal" href="../shelve/index.html#module-shelve" title="shelve: Persistent storage of arbitrary Python objects"><tt class="xref py py-mod docutils literal"><span class="pre">shelve</span></tt></a> would be easier to work with.</p>
<div class="highlight-python"><pre>$ python pickle_stream.py
WRITING: pickle (elkcip)
WRITING: cPickle (elkciPc)
WRITING: last (tsal)
READ: pickle (elkcip)
READ: cPickle (elkciPc)
READ: last (tsal)</pre>
</div>
<p>Besides storing data, pickles are very handy for inter-process
communication. For example, using <tt class="docutils literal"><span class="pre">os.fork()</span></tt> and <tt class="docutils literal"><span class="pre">os.pipe()</span></tt>, one
can establish worker processes that read job instructions from one
pipe and write the results to another pipe. The core code for managing
the worker pool and sending jobs in and receiving responses can be
reused, since the job and response objects don’t have to be of a
particular class. If you are using pipes or sockets, do not forget to
flush after dumping each object, to push the data through the
connection to the other end. See <a class="reference internal" href="../multiprocessing/index.html#module-multiprocessing" title="multiprocessing: Manage processes like threads."><tt class="xref py py-mod docutils literal"><span class="pre">multiprocessing</span></tt></a> if you don’t
want to write your own worker pool manager.</p>
</div>
<div class="section" id="problems-reconstructing-objects">
<h2>Problems Reconstructing Objects<a class="headerlink" href="#problems-reconstructing-objects" title="Permalink to this headline">¶</a></h2>
<p>When working with your own classes, you must ensure that the class being
pickled appears in the namespace of the process reading the pickle. Only the
data for the instance is pickled, not the class definition. The class name is
used to find the constructor to create the new object when unpickling. Take
this example, which writes instances of a class to a file:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span>
<span class="k">except</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="k">class</span> <span class="nc">SimpleObject</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="n">l</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="n">l</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name_backwards</span> <span class="o">=</span> <span class="s">''</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">l</span><span class="p">)</span>
<span class="k">return</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">data</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">SimpleObject</span><span class="p">(</span><span class="s">'pickle'</span><span class="p">))</span>
<span class="n">data</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">SimpleObject</span><span class="p">(</span><span class="s">'cPickle'</span><span class="p">))</span>
<span class="n">data</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">SimpleObject</span><span class="p">(</span><span class="s">'last'</span><span class="p">))</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">filename</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">except</span> <span class="ne">IndexError</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">RuntimeError</span><span class="p">(</span><span class="s">'Please specify a filename as an argument to </span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="n">out_s</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="c"># Write to the stream</span>
<span class="k">for</span> <span class="n">o</span> <span class="ow">in</span> <span class="n">data</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'WRITING: </span><span class="si">%s</span><span class="s"> (</span><span class="si">%s</span><span class="s">)'</span> <span class="o">%</span> <span class="p">(</span><span class="n">o</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">o</span><span class="o">.</span><span class="n">name_backwards</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">o</span><span class="p">,</span> <span class="n">out_s</span><span class="p">)</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">out_s</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>When run, the script creates a file based on the name given as
argument on the command line:</p>
<div class="highlight-python"><pre>$ python pickle_dump_to_file_1.py test.dat
WRITING: pickle (elkcip)
WRITING: cPickle (elkciPc)
WRITING: last (tsal)</pre>
</div>
<p>A simplistic attempt to load the resulting pickled objects fails:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span>
<span class="k">except</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">import</span> <span class="nn">pprint</span>
<span class="kn">from</span> <span class="nn">StringIO</span> <span class="kn">import</span> <span class="n">StringIO</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">filename</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">except</span> <span class="ne">IndexError</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">RuntimeError</span><span class="p">(</span><span class="s">'Please specify a filename as an argument to </span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="n">in_s</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s">'rb'</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="c"># Read the data</span>
<span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">o</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">in_s</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">EOFError</span><span class="p">:</span>
<span class="k">break</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'READ: </span><span class="si">%s</span><span class="s"> (</span><span class="si">%s</span><span class="s">)'</span> <span class="o">%</span> <span class="p">(</span><span class="n">o</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">o</span><span class="o">.</span><span class="n">name_backwards</span><span class="p">)</span>
<span class="k">finally</span><span class="p">:</span>
<span class="n">in_s</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>This version fails because there is no SimpleObject class available:</p>
<div class="highlight-python"><pre>$ python pickle_load_from_file_1.py test.dat
Traceback (most recent call last):
File "pickle_load_from_file_1.py", line 52, in <module>
o = pickle.load(in_s)
AttributeError: 'module' object has no attribute 'SimpleObject'</pre>
</div>
<p>The corrected version, which imports SimpleObject from the original
script, succeeds.</p>
<p>Add:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">pickle_dump_to_file_1</span> <span class="kn">import</span> <span class="n">SimpleObject</span>
</pre></div>
</div>
<p>to the end of the import list, then re-run the script:</p>
<div class="highlight-python"><pre>$ python pickle_load_from_file_2.py test.dat
READ: pickle (elkcip)
READ: cPickle (elkciPc)
READ: last (tsal)</pre>
</div>
<p>There are some special considerations when pickling data types with
values that cannot be pickled (sockets, file handles, database
connections, etc.). Classes that use values which cannot be pickled
can define <tt class="docutils literal"><span class="pre">__getstate__()</span></tt> and <tt class="docutils literal"><span class="pre">__setstate__()</span></tt> to return a
subset of the state of the instance to be pickled. New-style classes
can also define <tt class="docutils literal"><span class="pre">__getnewargs__()</span></tt>, which should return arguments to
be passed to the class memory allocator (<tt class="docutils literal"><span class="pre">C.__new__()</span></tt>). Use of
these features is covered in more detail in the standard library
documentation.</p>
</div>
<div class="section" id="circular-references">
<h2>Circular References<a class="headerlink" href="#circular-references" title="Permalink to this headline">¶</a></h2>
<p>The pickle protocol automatically handles circular references between
objects, so you don’t need to do anything special with complex data
structures. Consider the digraph:</p>
<p class="graphviz">
<img src="../_images/graphviz-66fc6f0c5ce5dbae5d0dae2b027dfd0d8edfdc38.png" alt="digraph pickle_example {
"root";
"root" -> "a";
"root" -> "b";
"a" -> "b";
"b" -> "a";
"b" -> "c";
"a" -> "a";
}" />
</p>
<p>Even though the graph includes several cycles, the correct structure
can be pickled and then reloaded.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">pickle</span>
<span class="k">class</span> <span class="nc">Node</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="sd">"""A simple digraph where each node knows about the other nodes</span>
<span class="sd"> it leads to.</span>
<span class="sd"> """</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connections</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">add_edge</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">node</span><span class="p">):</span>
<span class="s">"Create an edge between this node and the other."</span>
<span class="bp">self</span><span class="o">.</span><span class="n">connections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">iter</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">connections</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">preorder_traversal</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">seen</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">parent</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
<span class="sd">"""Generator function to yield the edges via a preorder traversal."""</span>
<span class="k">if</span> <span class="n">seen</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">seen</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="k">yield</span> <span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="n">root</span><span class="p">)</span>
<span class="k">if</span> <span class="n">root</span> <span class="ow">in</span> <span class="n">seen</span><span class="p">:</span>
<span class="k">return</span>
<span class="n">seen</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">root</span><span class="p">)</span>
<span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">root</span><span class="p">:</span>
<span class="k">for</span> <span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="n">subnode</span><span class="p">)</span> <span class="ow">in</span> <span class="n">preorder_traversal</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">seen</span><span class="p">,</span> <span class="n">root</span><span class="p">):</span>
<span class="k">yield</span> <span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="n">subnode</span><span class="p">)</span>
<span class="k">return</span>
<span class="k">def</span> <span class="nf">show_edges</span><span class="p">(</span><span class="n">root</span><span class="p">):</span>
<span class="s">"Print all of the edges in the graph."</span>
<span class="k">for</span> <span class="n">parent</span><span class="p">,</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">preorder_traversal</span><span class="p">(</span><span class="n">root</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">parent</span><span class="p">:</span>
<span class="k">continue</span>
<span class="k">print</span> <span class="s">'</span><span class="si">%5s</span><span class="s"> -> </span><span class="si">%2s</span><span class="s"> (</span><span class="si">%s</span><span class="s">)'</span> <span class="o">%</span> <span class="p">(</span><span class="n">parent</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">child</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="nb">id</span><span class="p">(</span><span class="n">child</span><span class="p">))</span>
<span class="c"># Set up the nodes.</span>
<span class="n">root</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span><span class="s">'root'</span><span class="p">)</span>
<span class="n">a</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span><span class="s">'a'</span><span class="p">)</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span><span class="s">'b'</span><span class="p">)</span>
<span class="n">c</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span><span class="s">'c'</span><span class="p">)</span>
<span class="c"># Add edges between them.</span>
<span class="n">root</span><span class="o">.</span><span class="n">add_edge</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="n">root</span><span class="o">.</span><span class="n">add_edge</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
<span class="n">a</span><span class="o">.</span><span class="n">add_edge</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
<span class="n">b</span><span class="o">.</span><span class="n">add_edge</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="n">b</span><span class="o">.</span><span class="n">add_edge</span><span class="p">(</span><span class="n">c</span><span class="p">)</span>
<span class="n">a</span><span class="o">.</span><span class="n">add_edge</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'ORIGINAL GRAPH:'</span>
<span class="n">show_edges</span><span class="p">(</span><span class="n">root</span><span class="p">)</span>
<span class="c"># Pickle and unpickle the graph to create</span>
<span class="c"># a new set of nodes.</span>
<span class="n">dumped</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">root</span><span class="p">)</span>
<span class="n">reloaded</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">dumped</span><span class="p">)</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'RELOADED GRAPH:'</span>
<span class="n">show_edges</span><span class="p">(</span><span class="n">reloaded</span><span class="p">)</span>
</pre></div>
</div>
<p>The reloaded nodes are not the same object, but the relationship
between the nodes is maintained and only one copy of the object with
multiple reference is reloaded. Both of these statements can be
verified by examining the <tt class="docutils literal"><span class="pre">id()</span></tt> values for the nodes before and
after being passed through pickle.</p>
<div class="highlight-python"><pre>$ python pickle_cycle.py
ORIGINAL GRAPH:
root -> a (4299677584)
a -> b (4299677648)
b -> a (4299677584)
b -> c (4299718736)
a -> a (4299677584)
root -> b (4299677648)
RELOADED GRAPH:
root -> a (4299718864)
a -> b (4299718928)
b -> a (4299718864)
b -> c (4299718992)
a -> a (4299718864)
root -> b (4299718928)</pre>
</div>
<div class="admonition-see-also admonition seealso">
<p class="first admonition-title">See also</p>
<dl class="docutils">
<dt><a class="reference external" href="http://docs.python.org/lib/module-pickle.html">pickle</a></dt>
<dd>Standard library documentation for this module.</dd>
<dt><a class="reference internal" href="../shelve/index.html#module-shelve" title="shelve: Persistent storage of arbitrary Python objects"><tt class="xref py py-mod docutils literal"><span class="pre">shelve</span></tt></a></dt>
<dd>The shelve module.</dd>
<dt><a class="reference external" href="http://peadrop.com/blog/2007/06/18/pickle-an-interesting-stack-language/">Pickle: An interesting stack language.</a></dt>
<dd>by Alexandre Vassalotti</dd>
</dl>
<p class="last"><a class="reference internal" href="../articles/data_persistence.html#article-data-persistence"><em>Data Persistence and Exchange</em></a></p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="../shelve/index.html" title="shelve – Persistent storage of arbitrary Python objects"
>next</a> |</li>
<li class="right" >
<a href="../gdbm/index.html" title="gdbm – GNU’s version of the dbm library"
>previous</a> |</li>
<li><a href="../contents.html">PyMOTW</a> »</li>
<li><a href="../persistence.html" >Data Persistence</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright Doug Hellmann.
Last updated on Oct 24, 2010.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
<br/><a href="http://creativecommons.org/licenses/by-nc-sa/3.0/us/" rel="license"><img alt="Creative Commons License" style="border-width:0" src="http://i.creativecommons.org/l/by-nc-sa/3.0/us/88x31.png"/></a>
</div>
</body>
</html>