<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>json – JavaScript Object Notation Serializer — Python Module of the Week</title>
<link rel="stylesheet" href="../_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '1.132',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="author" title="About these documents" href="../about.html" />
<link rel="top" title="Python Module of the Week" href="../index.html" />
<link rel="up" title="Internet Data Handling" href="../internet_data.html" />
<link rel="next" title="mailbox – Access and manipulate email archives" href="../mailbox/index.html" />
<link rel="prev" title="base64 – Encode binary data into ASCII characters" href="../base64/index.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="../mailbox/index.html" title="mailbox – Access and manipulate email archives"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="../base64/index.html" title="base64 – Encode binary data into ASCII characters"
accesskey="P">previous</a> |</li>
<li><a href="../contents.html">PyMOTW</a> »</li>
<li><a href="../internet_data.html" accesskey="U">Internet Data Handling</a> »</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="../contents.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">json – JavaScript Object Notation Serializer</a><ul>
<li><a class="reference internal" href="#encoding-and-decoding-simple-data-types">Encoding and Decoding Simple Data Types</a></li>
<li><a class="reference internal" href="#human-consumable-vs-compact-output">Human-consumable vs. Compact Output</a></li>
<li><a class="reference internal" href="#encoding-dictionaries">Encoding Dictionaries</a></li>
<li><a class="reference internal" href="#working-with-your-own-types">Working with Your Own Types</a></li>
<li><a class="reference internal" href="#encoder-and-decoder-classes">Encoder and Decoder Classes</a></li>
<li><a class="reference internal" href="#working-with-streams-and-files">Working with Streams and Files</a></li>
<li><a class="reference internal" href="#mixed-data-streams">Mixed Data Streams</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="../base64/index.html"
title="previous chapter">base64 – Encode binary data into ASCII characters</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="../mailbox/index.html"
title="next chapter">mailbox – Access and manipulate email archives</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/json/index.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="module-json">
<span id="json-javascript-object-notation-serializer"></span><h1>json – JavaScript Object Notation Serializer<a class="headerlink" href="#module-json" title="Permalink to this headline">¶</a></h1>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field"><th class="field-name">Purpose:</th><td class="field-body">Encode Python objects as JSON strings, and decode JSON strings into Python objects.</td>
</tr>
<tr class="field"><th class="field-name">Python Version:</th><td class="field-body">2.6</td>
</tr>
</tbody>
</table>
<p>The <a class="reference internal" href="#module-json" title="json: JavaScript Object Notation Serializer"><tt class="xref py py-mod docutils literal"><span class="pre">json</span></tt></a> module provides an API similar to <a class="reference internal" href="../pickle/index.html#module-pickle" title="pickle: Python object serialization"><tt class="xref py py-mod docutils literal"><span class="pre">pickle</span></tt></a> for
converting in-memory Python objects to a serialized representation
known as <a class="reference external" href="http://json.org/">JavaScript Object Notation</a> (JSON). Unlike pickle, JSON
has the benefit of having implementations in many languages
(especially JavaScript), making it suitable for inter-application
communication. JSON is probably most widely used for communicating
between the web server and client in an AJAX application, but is not
limited to that problem domain.</p>
<div class="section" id="encoding-and-decoding-simple-data-types">
<h2>Encoding and Decoding Simple Data Types<a class="headerlink" href="#encoding-and-decoding-simple-data-types" title="Permalink to this headline">¶</a></h2>
<p>The encoder understands Python’s native types by default (string,
unicode, int, float, list, tuple, dict).</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="k">print</span> <span class="s">'DATA:'</span><span class="p">,</span> <span class="nb">repr</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="n">data_string</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'JSON:'</span><span class="p">,</span> <span class="n">data_string</span>
</pre></div>
</div>
<p>Values are encoded in a manner very similar to Python’s <tt class="docutils literal"><span class="pre">repr()</span></tt>
output.</p>
<div class="highlight-python"><pre>$ python json_simple_types.py
DATA: [{'a': 'A', 'c': 3.0, 'b': (2, 4)}]
JSON: [{"a": "A", "c": 3.0, "b": [2, 4]}]</pre>
</div>
<p>Encoding, then re-decoding may not give exactly the same type of
object.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="n">data_string</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'ENCODED:'</span><span class="p">,</span> <span class="n">data_string</span>
<span class="n">decoded</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">data_string</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'DECODED:'</span><span class="p">,</span> <span class="n">decoded</span>
<span class="k">print</span> <span class="s">'ORIGINAL:'</span><span class="p">,</span> <span class="nb">type</span><span class="p">(</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s">'b'</span><span class="p">])</span>
<span class="k">print</span> <span class="s">'DECODED :'</span><span class="p">,</span> <span class="nb">type</span><span class="p">(</span><span class="n">decoded</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s">'b'</span><span class="p">])</span>
</pre></div>
</div>
<p>In particular, strings are converted to unicode and tuples become
lists.</p>
<div class="highlight-python"><pre>$ python json_simple_types_decode.py
ENCODED: [{"a": "A", "c": 3.0, "b": [2, 4]}]
DECODED: [{'a': 'A', 'c': 3.0, 'b': [2, 4]}]
ORIGINAL: <type 'tuple'>
DECODED : <type 'list'></pre>
</div>
</div>
<div class="section" id="human-consumable-vs-compact-output">
<h2>Human-consumable vs. Compact Output<a class="headerlink" href="#human-consumable-vs-compact-output" title="Permalink to this headline">¶</a></h2>
<p>Another benefit of JSON over pickle is that the results are
human-readable. The <tt class="docutils literal"><span class="pre">dumps()</span></tt> function accepts several arguments to
make the output even nicer. For example, <tt class="docutils literal"><span class="pre">sort_keys</span></tt> tells the
encoder to output the keys of a dictionary in sorted, instead of
random, order.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="k">print</span> <span class="s">'DATA:'</span><span class="p">,</span> <span class="nb">repr</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="n">unsorted</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'JSON:'</span><span class="p">,</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'SORT:'</span><span class="p">,</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">first</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">second</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'UNSORTED MATCH:'</span><span class="p">,</span> <span class="n">unsorted</span> <span class="o">==</span> <span class="n">first</span>
<span class="k">print</span> <span class="s">'SORTED MATCH :'</span><span class="p">,</span> <span class="n">first</span> <span class="o">==</span> <span class="n">second</span>
</pre></div>
</div>
<p>Sorting makes it easier to scan the results by eye, and also makes it
possible to compare JSON output in tests.</p>
<div class="highlight-python"><pre>$ python json_sort_keys.py
DATA: [{'a': 'A', 'c': 3.0, 'b': (2, 4)}]
JSON: [{"a": "A", "c": 3.0, "b": [2, 4]}]
SORT: [{"a": "A", "b": [2, 4], "c": 3.0}]
UNSORTED MATCH: False
SORTED MATCH : True</pre>
</div>
<p>For highly-nested data structures, you will want to specify a value
for <tt class="docutils literal"><span class="pre">indent</span></tt>, so the output is formatted nicely as well.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="k">print</span> <span class="s">'DATA:'</span><span class="p">,</span> <span class="nb">repr</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'NORMAL:'</span><span class="p">,</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'INDENT:'</span><span class="p">,</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">sort_keys</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
</pre></div>
</div>
<p>When indent is a non-negative integer, the output more closely
resembles that of <a class="reference internal" href="../pprint/index.html#module-pprint" title="pprint: Pretty-print data structures"><tt class="xref py py-mod docutils literal"><span class="pre">pprint</span></tt></a>, with leading spaces for each level of
the data structure matching the indent level.</p>
<div class="highlight-python"><pre>$ python json_indent.py
DATA: [{'a': 'A', 'c': 3.0, 'b': (2, 4)}]
NORMAL: [{"a": "A", "b": [2, 4], "c": 3.0}]
INDENT: [
{
"a": "A",
"b": [
2,
4
],
"c": 3.0
}
]</pre>
</div>
<p>Verbose output like this increases the number of bytes needed to
transmit the same amount of data, however, so it isn’t the sort of
thing you necessarily want to use in a production environment. In
fact, you may want to adjust the settings for separating data in the
encoded output to make it even more compact than the default.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="k">print</span> <span class="s">'DATA:'</span><span class="p">,</span> <span class="nb">repr</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'repr(data) :'</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="nb">repr</span><span class="p">(</span><span class="n">data</span><span class="p">))</span>
<span class="k">print</span> <span class="s">'dumps(data) :'</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">))</span>
<span class="k">print</span> <span class="s">'dumps(data, indent=2) :'</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">indent</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span>
<span class="k">print</span> <span class="s">'dumps(data, separators):'</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">separators</span><span class="o">=</span><span class="p">(</span><span class="s">','</span><span class="p">,</span><span class="s">':'</span><span class="p">)))</span>
</pre></div>
</div>
<p>The <tt class="docutils literal"><span class="pre">separators</span></tt> argument to <tt class="docutils literal"><span class="pre">dumps()</span></tt> should be a tuple
containing the strings to separate items in a list and keys from
values in a dictionary. The default is <tt class="docutils literal"><span class="pre">(',</span> <span class="pre">',</span> <span class="pre">':</span> <span class="pre">')</span></tt>. By removing
the whitespace, we can produce a more compact output.</p>
<div class="highlight-python"><pre>$ python json_compact_encoding.py
DATA: [{'a': 'A', 'c': 3.0, 'b': (2, 4)}]
repr(data) : 35
dumps(data) : 35
dumps(data, indent=2) : 76
dumps(data, separators): 29</pre>
</div>
</div>
<div class="section" id="encoding-dictionaries">
<h2>Encoding Dictionaries<a class="headerlink" href="#encoding-dictionaries" title="Permalink to this headline">¶</a></h2>
<p>The JSON format expects the keys to a dictionary to be strings. If
you have other types as keys in your dictionary, trying to encode the
object will produce a <a class="reference internal" href="../exceptions/index.html#exceptions-valueerror"><em>ValueError</em></a>. One way
to work around that limitation is to skip over non-string keys using
the <tt class="docutils literal"><span class="pre">skipkeys</span></tt> argument:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span><span class="p">,</span> <span class="p">(</span><span class="s">'d'</span><span class="p">,):</span><span class="s">'D tuple'</span> <span class="p">}</span> <span class="p">]</span>
<span class="k">print</span> <span class="s">'First attempt'</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">print</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">,</span> <span class="n">err</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'ERROR:'</span><span class="p">,</span> <span class="n">err</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'Second attempt'</span>
<span class="k">print</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">skipkeys</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
</pre></div>
</div>
<p>Rather than raising an exception, the non-string key is simply
ignored.</p>
<div class="highlight-python"><pre>$ python json_skipkeys.py
First attempt
ERROR: keys must be a string
Second attempt
[{"a": "A", "c": 3.0, "b": [2, 4]}]</pre>
</div>
</div>
<div class="section" id="working-with-your-own-types">
<h2>Working with Your Own Types<a class="headerlink" href="#working-with-your-own-types" title="Permalink to this headline">¶</a></h2>
<p>All of the examples so far have used Pythons built-in types because
those are supported by <a class="reference internal" href="#module-json" title="json: JavaScript Object Notation Serializer"><tt class="xref py py-mod docutils literal"><span class="pre">json</span></tt></a> natively. It isn’t uncommon, of
course, to have your own types that you want to be able to encode as
well. There are two ways to do that.</p>
<p>First, we’ll need a class to encode:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">class</span> <span class="nc">MyObj</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">s</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">s</span> <span class="o">=</span> <span class="n">s</span>
<span class="k">def</span> <span class="nf">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s">'<MyObj(</span><span class="si">%s</span><span class="s">)>'</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">s</span>
</pre></div>
</div>
<p>The simple way of encoding a <tt class="docutils literal"><span class="pre">MyObj</span></tt> instance is to define a
function to convert an unknown type to a known type. You don’t have
to do the encoding yourself, just convert one object to another.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">json_myobj</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">json_myobj</span><span class="o">.</span><span class="n">MyObj</span><span class="p">(</span><span class="s">'instance value goes here'</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'First attempt'</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">print</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">TypeError</span><span class="p">,</span> <span class="n">err</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'ERROR:'</span><span class="p">,</span> <span class="n">err</span>
<span class="k">def</span> <span class="nf">convert_to_builtin_type</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="k">print</span> <span class="s">'default('</span><span class="p">,</span> <span class="nb">repr</span><span class="p">(</span><span class="n">obj</span><span class="p">),</span> <span class="s">')'</span>
<span class="c"># Convert objects to a dictionary of their representation</span>
<span class="n">d</span> <span class="o">=</span> <span class="p">{</span> <span class="s">'__class__'</span><span class="p">:</span><span class="n">obj</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span>
<span class="s">'__module__'</span><span class="p">:</span><span class="n">obj</span><span class="o">.</span><span class="n">__module__</span><span class="p">,</span>
<span class="p">}</span>
<span class="n">d</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="n">__dict__</span><span class="p">)</span>
<span class="k">return</span> <span class="n">d</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'With default'</span>
<span class="k">print</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="n">convert_to_builtin_type</span><span class="p">)</span>
</pre></div>
</div>
<p>In <tt class="docutils literal"><span class="pre">convert_to_builtin_type()</span></tt>, instances of classes not recognized
by <a class="reference internal" href="#module-json" title="json: JavaScript Object Notation Serializer"><tt class="xref py py-mod docutils literal"><span class="pre">json</span></tt></a> are converted to dictionaries with enough information
to re-create the object if a program has access to the Python modules
necessary.</p>
<div class="highlight-python"><pre>$ python json_dump_default.py
First attempt
ERROR: <MyObj(instance value goes here)> is not JSON serializable
With default
default( <MyObj(instance value goes here)> )
{"s": "instance value goes here", "__module__": "json_myobj", "__class__": "MyObj"}</pre>
</div>
<p>To decode the results and create a <tt class="docutils literal"><span class="pre">MyObj</span></tt> instance, we need to tie
in to the decoder so we can import the class from the module and
create the instance. For that, we use the <tt class="docutils literal"><span class="pre">object_hook</span></tt> argument to
<tt class="docutils literal"><span class="pre">loads()</span></tt>.</p>
<p>The <tt class="docutils literal"><span class="pre">object_hook</span></tt> is called for each dictionary decoded from the
incoming data stream, giving us a chance to convert the dictionary to
another type of object. The hook function should return the object it
wants the calling application to receive instead of the dictionary.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="k">def</span> <span class="nf">dict_to_object</span><span class="p">(</span><span class="n">d</span><span class="p">):</span>
<span class="k">if</span> <span class="s">'__class__'</span> <span class="ow">in</span> <span class="n">d</span><span class="p">:</span>
<span class="n">class_name</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s">'__class__'</span><span class="p">)</span>
<span class="n">module_name</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s">'__module__'</span><span class="p">)</span>
<span class="n">module</span> <span class="o">=</span> <span class="nb">__import__</span><span class="p">(</span><span class="n">module_name</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'MODULE:'</span><span class="p">,</span> <span class="n">module</span>
<span class="n">class_</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">module</span><span class="p">,</span> <span class="n">class_name</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'CLASS:'</span><span class="p">,</span> <span class="n">class_</span>
<span class="n">args</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span> <span class="p">(</span><span class="n">key</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s">'ascii'</span><span class="p">),</span> <span class="n">value</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
<span class="k">print</span> <span class="s">'INSTANCE ARGS:'</span><span class="p">,</span> <span class="n">args</span>
<span class="n">inst</span> <span class="o">=</span> <span class="n">class_</span><span class="p">(</span><span class="o">**</span><span class="n">args</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">inst</span> <span class="o">=</span> <span class="n">d</span>
<span class="k">return</span> <span class="n">inst</span>
<span class="n">encoded_object</span> <span class="o">=</span> <span class="s">'[{"s": "instance value goes here", "__module__": "json_myobj", "__class__": "MyObj"}]'</span>
<span class="n">myobj_instance</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">encoded_object</span><span class="p">,</span> <span class="n">object_hook</span><span class="o">=</span><span class="n">dict_to_object</span><span class="p">)</span>
<span class="k">print</span> <span class="n">myobj_instance</span>
</pre></div>
</div>
<p>Since <a class="reference internal" href="#module-json" title="json: JavaScript Object Notation Serializer"><tt class="xref py py-mod docutils literal"><span class="pre">json</span></tt></a> converts string values to unicode objects, we need
to re-encode them as ASCII strings before using them as keyword
arguments to the class constructor.</p>
<div class="highlight-python"><pre>$ python json_load_object_hook.py
MODULE: <module 'json_myobj' from '/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/json/json_myobj.pyc'>
CLASS: <class 'json_myobj.MyObj'>
INSTANCE ARGS: {'s': 'instance value goes here'}
[<MyObj(instance value goes here)>]</pre>
</div>
<p>Similar hooks are available for the built-in types integers
(<tt class="docutils literal"><span class="pre">parse_int</span></tt>), floating point numbers (<tt class="docutils literal"><span class="pre">parse_float</span></tt>), and
constants (<tt class="docutils literal"><span class="pre">parse_constant</span></tt>).</p>
</div>
<div class="section" id="encoder-and-decoder-classes">
<h2>Encoder and Decoder Classes<a class="headerlink" href="#encoder-and-decoder-classes" title="Permalink to this headline">¶</a></h2>
<p>Besides the convenience functions we have already examined, the
<a class="reference internal" href="#module-json" title="json: JavaScript Object Notation Serializer"><tt class="xref py py-mod docutils literal"><span class="pre">json</span></tt></a> module provides classes for encoding and decoding. When
using the classes directly, you have access to extra APIs and can
create subclasses to customize their behavior.</p>
<p>The JSONEncoder provides an iterable interface for producing “chunks”
of encoded data, making it easier for you to write to files or network
sockets without having to represent an entire data structure in
memory.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="n">encoder</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">JSONEncoder</span><span class="p">()</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="k">for</span> <span class="n">part</span> <span class="ow">in</span> <span class="n">encoder</span><span class="o">.</span><span class="n">iterencode</span><span class="p">(</span><span class="n">data</span><span class="p">):</span>
<span class="k">print</span> <span class="s">'PART:'</span><span class="p">,</span> <span class="n">part</span>
</pre></div>
</div>
<p>As you can see, the output is generated in logical units, rather than
being based on any size value.</p>
<div class="highlight-python"><pre>$ python json_encoder_iterable.py
PART: [
PART: {
PART: "a"
PART: :
PART: "A"
PART: ,
PART: "c"
PART: :
PART: 3.0
PART: ,
PART: "b"
PART: :
PART: [2
PART: , 4
PART: ]
PART: }
PART: ]</pre>
</div>
<p>The <tt class="docutils literal"><span class="pre">encode()</span></tt> method is basically equivalent to
<tt class="docutils literal"><span class="pre">''.join(encoder.iterencode())</span></tt>, with some extra error checking up
front.</p>
<p>To encode arbitrary objects, we can override the <tt class="docutils literal"><span class="pre">default()</span></tt> method
with an implementation similar to what we used above in
<tt class="docutils literal"><span class="pre">convert_to_builtin_type()</span></tt>.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">json_myobj</span>
<span class="k">class</span> <span class="nc">MyEncoder</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">JSONEncoder</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">default</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>
<span class="k">print</span> <span class="s">'default('</span><span class="p">,</span> <span class="nb">repr</span><span class="p">(</span><span class="n">obj</span><span class="p">),</span> <span class="s">')'</span>
<span class="c"># Convert objects to a dictionary of their representation</span>
<span class="n">d</span> <span class="o">=</span> <span class="p">{</span> <span class="s">'__class__'</span><span class="p">:</span><span class="n">obj</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span>
<span class="s">'__module__'</span><span class="p">:</span><span class="n">obj</span><span class="o">.</span><span class="n">__module__</span><span class="p">,</span>
<span class="p">}</span>
<span class="n">d</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="n">__dict__</span><span class="p">)</span>
<span class="k">return</span> <span class="n">d</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">json_myobj</span><span class="o">.</span><span class="n">MyObj</span><span class="p">(</span><span class="s">'internal data'</span><span class="p">)</span>
<span class="k">print</span> <span class="n">obj</span>
<span class="k">print</span> <span class="n">MyEncoder</span><span class="p">()</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
</pre></div>
</div>
<p>The output is the same as the previous implementation.</p>
<div class="highlight-python"><pre>$ python json_encoder_default.py
<MyObj(internal data)>
default( <MyObj(internal data)> )
{"s": "internal data", "__module__": "json_myobj", "__class__": "MyObj"}</pre>
</div>
<p>Decoding text, then converting the dictionary into an object takes a
little more work to set up than our previous implementation, but not
much.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="k">class</span> <span class="nc">MyDecoder</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">JSONDecoder</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">json</span><span class="o">.</span><span class="n">JSONDecoder</span><span class="o">.</span><span class="n">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">object_hook</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dict_to_object</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">dict_to_object</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">d</span><span class="p">):</span>
<span class="k">if</span> <span class="s">'__class__'</span> <span class="ow">in</span> <span class="n">d</span><span class="p">:</span>
<span class="n">class_name</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s">'__class__'</span><span class="p">)</span>
<span class="n">module_name</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s">'__module__'</span><span class="p">)</span>
<span class="n">module</span> <span class="o">=</span> <span class="nb">__import__</span><span class="p">(</span><span class="n">module_name</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'MODULE:'</span><span class="p">,</span> <span class="n">module</span>
<span class="n">class_</span> <span class="o">=</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">module</span><span class="p">,</span> <span class="n">class_name</span><span class="p">)</span>
<span class="k">print</span> <span class="s">'CLASS:'</span><span class="p">,</span> <span class="n">class_</span>
<span class="n">args</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span> <span class="p">(</span><span class="n">key</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s">'ascii'</span><span class="p">),</span> <span class="n">value</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
<span class="k">print</span> <span class="s">'INSTANCE ARGS:'</span><span class="p">,</span> <span class="n">args</span>
<span class="n">inst</span> <span class="o">=</span> <span class="n">class_</span><span class="p">(</span><span class="o">**</span><span class="n">args</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">inst</span> <span class="o">=</span> <span class="n">d</span>
<span class="k">return</span> <span class="n">inst</span>
<span class="n">encoded_object</span> <span class="o">=</span> <span class="s">'[{"s": "instance value goes here", "__module__": "json_myobj", "__class__": "MyObj"}]'</span>
<span class="n">myobj_instance</span> <span class="o">=</span> <span class="n">MyDecoder</span><span class="p">()</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="n">encoded_object</span><span class="p">)</span>
<span class="k">print</span> <span class="n">myobj_instance</span>
</pre></div>
</div>
<p>And the output is the same as the earlier example.</p>
<div class="highlight-python"><pre>$ python json_decoder_object_hook.py
MODULE: <module 'json_myobj' from '/Users/dhellmann/Documents/PyMOTW/src/PyMOTW/json/json_myobj.pyc'>
CLASS: <class 'json_myobj.MyObj'>
INSTANCE ARGS: {'s': 'instance value goes here'}
[<MyObj(instance value goes here)>]</pre>
</div>
</div>
<div class="section" id="working-with-streams-and-files">
<h2>Working with Streams and Files<a class="headerlink" href="#working-with-streams-and-files" title="Permalink to this headline">¶</a></h2>
<p>In all of the examples so far, we have assumed that we could (and
should) hold the encoded version of the entire data structure in
memory at one time. With large data structures it may be preferable
to write the encoding directly to a file-like object. The convenience
functions <tt class="docutils literal"><span class="pre">load()</span></tt> and <tt class="docutils literal"><span class="pre">dump()</span></tt> accept references to a file-like
object to use for reading or writing.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">tempfile</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">[</span> <span class="p">{</span> <span class="s">'a'</span><span class="p">:</span><span class="s">'A'</span><span class="p">,</span> <span class="s">'b'</span><span class="p">:(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">),</span> <span class="s">'c'</span><span class="p">:</span><span class="mf">3.0</span> <span class="p">}</span> <span class="p">]</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">tempfile</span><span class="o">.</span><span class="n">NamedTemporaryFile</span><span class="p">(</span><span class="n">mode</span><span class="o">=</span><span class="s">'w+'</span><span class="p">)</span>
<span class="n">json</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">f</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="k">print</span> <span class="nb">open</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="s">'r'</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</pre></div>
</div>
<p>A socket would work in much the same way as the normal file handle
used here.</p>
<div class="highlight-python"><pre>$ python json_dump_file.py
[{"a": "A", "c": 3.0, "b": [2, 4]}]</pre>
</div>
<p>Although it isn’t optimized to read only part of the data at a time,
the <tt class="docutils literal"><span class="pre">load()</span></tt> function still offers the benefit of encapsulating the
logic of generating objects from stream input.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">tempfile</span>
<span class="n">f</span> <span class="o">=</span> <span class="n">tempfile</span><span class="o">.</span><span class="n">NamedTemporaryFile</span><span class="p">(</span><span class="n">mode</span><span class="o">=</span><span class="s">'w+'</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">'[{"a": "A", "c": 3.0, "b": [2, 4]}]'</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="n">f</span><span class="o">.</span><span class="n">seek</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="k">print</span> <span class="n">json</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python"><pre>$ python json_load_file.py
[{'a': 'A', 'c': 3.0, 'b': [2, 4]}]</pre>
</div>
</div>
<div class="section" id="mixed-data-streams">
<h2>Mixed Data Streams<a class="headerlink" href="#mixed-data-streams" title="Permalink to this headline">¶</a></h2>
<p>The JSONDecoder includes the <tt class="docutils literal"><span class="pre">raw_decode()</span></tt> method for decoding a
data structure followed by more data, such as JSON data with trailing
text. The return value is the object created by decoding the input
data, and an index into that data indicating where decoding left off.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">import</span> <span class="nn">json</span>
<span class="n">decoder</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">JSONDecoder</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">get_decoded_and_remainder</span><span class="p">(</span><span class="n">input_data</span><span class="p">):</span>
<span class="n">obj</span><span class="p">,</span> <span class="n">end</span> <span class="o">=</span> <span class="n">decoder</span><span class="o">.</span><span class="n">raw_decode</span><span class="p">(</span><span class="n">input_data</span><span class="p">)</span>
<span class="n">remaining</span> <span class="o">=</span> <span class="n">input_data</span><span class="p">[</span><span class="n">end</span><span class="p">:]</span>
<span class="k">return</span> <span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">remaining</span><span class="p">)</span>
<span class="n">encoded_object</span> <span class="o">=</span> <span class="s">'[{"a": "A", "c": 3.0, "b": [2, 4]}]'</span>
<span class="n">extra_text</span> <span class="o">=</span> <span class="s">'This text is not JSON.'</span>
<span class="k">print</span> <span class="s">'JSON first:'</span>
<span class="n">obj</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">remaining</span> <span class="o">=</span> <span class="n">get_decoded_and_remainder</span><span class="p">(</span><span class="s">' '</span><span class="o">.</span><span class="n">join</span><span class="p">([</span><span class="n">encoded_object</span><span class="p">,</span> <span class="n">extra_text</span><span class="p">]))</span>
<span class="k">print</span> <span class="s">'Object :'</span><span class="p">,</span> <span class="n">obj</span>
<span class="k">print</span> <span class="s">'End of parsed input :'</span><span class="p">,</span> <span class="n">end</span>
<span class="k">print</span> <span class="s">'Remaining text :'</span><span class="p">,</span> <span class="nb">repr</span><span class="p">(</span><span class="n">remaining</span><span class="p">)</span>
<span class="k">print</span>
<span class="k">print</span> <span class="s">'JSON embedded:'</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">obj</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">remaining</span> <span class="o">=</span> <span class="n">get_decoded_and_remainder</span><span class="p">(</span>
<span class="s">' '</span><span class="o">.</span><span class="n">join</span><span class="p">([</span><span class="n">extra_text</span><span class="p">,</span> <span class="n">encoded_object</span><span class="p">,</span> <span class="n">extra_text</span><span class="p">])</span>
<span class="p">)</span>
<span class="k">except</span> <span class="ne">ValueError</span><span class="p">,</span> <span class="n">err</span><span class="p">:</span>
<span class="k">print</span> <span class="s">'ERROR:'</span><span class="p">,</span> <span class="n">err</span>
</pre></div>
</div>
<p>Unfortunately, this only works if the object appears at the beginning
of the input.</p>
<div class="highlight-python"><pre>$ python json_mixed_data.py
JSON first:
Object : [{'a': 'A', 'c': 3.0, 'b': [2, 4]}]
End of parsed input : 35
Remaining text : ' This text is not JSON.'
JSON embedded:
ERROR: No JSON object could be decoded</pre>
</div>
<div class="admonition-see-also admonition seealso">
<p class="first admonition-title">See also</p>
<dl class="last docutils">
<dt><a class="reference external" href="http://docs.python.org/library/json.html">json</a></dt>
<dd>The standard library documentation for this module.</dd>
<dt><a class="reference external" href="http://json.org/">JavaScript Object Notation</a></dt>
<dd>JSON home, with documentation and implementations in other languages.</dd>
<dt><a class="reference external" href="http://code.google.com/p/simplejson/">http://code.google.com/p/simplejson/</a></dt>
<dd>simplejson, from Bob Ippolito, et al, is the externally
maintained development version of the json library included
with Python 2.6 and Python 3.0. It maintains backwards
compatibility with Python 2.4 and Python 2.5.</dd>
<dt><a class="reference external" href="http://code.google.com/p/jsonpickle/">jsonpickle</a></dt>
<dd>jsonpickle allows for any Python object to be serialized into JSON.</dd>
<dt><a class="reference internal" href="../articles/data_persistence.html#article-data-persistence"><em>Data Persistence and Exchange</em></a></dt>
<dd>Other examples of storing data from Python programs.</dd>
</dl>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="../mailbox/index.html" title="mailbox – Access and manipulate email archives"
>next</a> |</li>
<li class="right" >
<a href="../base64/index.html" title="base64 – Encode binary data into ASCII characters"
>previous</a> |</li>
<li><a href="../contents.html">PyMOTW</a> »</li>
<li><a href="../internet_data.html" >Internet Data Handling</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright Doug Hellmann.
Last updated on Oct 24, 2010.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
<br/><a href="http://creativecommons.org/licenses/by-nc-sa/3.0/us/" rel="license"><img alt="Creative Commons License" style="border-width:0" src="http://i.creativecommons.org/l/by-nc-sa/3.0/us/88x31.png"/></a>
</div>
</body>
</html>