DESCRIPTION.rst 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226
  1. Python JSONPath RW
  2. ==================
  3. https://github.com/kennknowles/python-jsonpath-rw
  4. |Build Status| |Test coverage| |PyPi version| |PyPi downloads|
  5. This library provides a robust and significantly extended implementation
  6. of JSONPath for Python. It is tested with Python 2.6, 2.7, 3.2, 3.3.
  7. *(On travis-ci there is a segfault when running the tests with pypy; I don't think the problem lies with this library)*.
  8. This library differs from other JSONPath implementations in that it is a
  9. full *language* implementation, meaning the JSONPath expressions are
  10. first class objects, easy to analyze, transform, parse, print, and
  11. extend. (You can also execute them :-)
  12. Quick Start
  13. -----------
  14. To install, use pip:
  15. ::
  16. $ pip install jsonpath-rw
  17. Then:
  18. .. code:: python
  19. $ python
  20. >>> from jsonpath_rw import jsonpath, parse
  21. # A robust parser, not just a regex. (Makes powerful extensions possible; see below)
  22. >>> jsonpath_expr = parse('foo[*].baz')
  23. # Extracting values is easy
  24. >>> [match.value for match in jsonpath_expr.find({'foo': [{'baz': 1}, {'baz': 2}]})]
  25. [1, 2]
  26. # Matches remember where they came from
  27. >>> [str(match.full_path) for match in jsonpath_expr.find({'foo': [{'baz': 1}, {'baz': 2}]})]
  28. ['foo.[0].baz', 'foo.[1].baz']
  29. # And this can be useful for automatically providing ids for bits of data that do not have them (currently a global switch)
  30. >>> jsonpath.auto_id_field = 'id'
  31. >>> [match.value for match in parse('foo[*].id').find({'foo': [{'id': 'bizzle'}, {'baz': 3}]})]
  32. ['foo.bizzle', 'foo.[1]']
  33. # A handy extension: named operators like `parent`
  34. >>> [match.value for match in parse('a.*.b.`parent`.c').find({'a': {'x': {'b': 1, 'c': 'number one'}, 'y': {'b': 2, 'c': 'number two'}}})]
  35. ['number two', 'number one']
  36. # You can also build expressions directly quite easily
  37. >>> from jsonpath_rw.jsonpath import Fields
  38. >>> from jsonpath_rw.jsonpath import Slice
  39. >>> jsonpath_expr_direct = Fields('foo').child(Slice('*')).child(Fields('baz')) # This is equivalent
  40. JSONPath Syntax
  41. ---------------
  42. The JSONPath syntax supported by this library includes some additional
  43. features and omits some problematic features (those that make it
  44. unportable). In particular, some new operators such as ``|`` and
  45. ``where`` are available, and parentheses are used for grouping not for
  46. callbacks into Python, since with these changes the language is not
  47. trivially associative. Also, fields may be quoted whether or not they
  48. are contained in brackets.
  49. Atomic expressions:
  50. +-----------------------+---------------------------------------------------------------------------------------------+
  51. | Syntax | Meaning |
  52. +=======================+=============================================================================================+
  53. | ``$`` | The root object |
  54. +-----------------------+---------------------------------------------------------------------------------------------+
  55. | ```this``` | The "current" object. |
  56. +-----------------------+---------------------------------------------------------------------------------------------+
  57. | ```foo``` | More generally, this syntax allows "named operators" to extend JSONPath is arbitrary ways |
  58. +-----------------------+---------------------------------------------------------------------------------------------+
  59. | *field* | Specified field(s), described below |
  60. +-----------------------+---------------------------------------------------------------------------------------------+
  61. | ``[`` *field* ``]`` | Same as *field* |
  62. +-----------------------+---------------------------------------------------------------------------------------------+
  63. | ``[`` *idx* ``]`` | Array access, described below (this is always unambiguous with field access) |
  64. +-----------------------+---------------------------------------------------------------------------------------------+
  65. Jsonpath operators:
  66. +-------------------------------------+------------------------------------------------------------------------------------+
  67. | Syntax | Meaning |
  68. +=====================================+====================================================================================+
  69. | *jsonpath1* ``.`` *jsonpath2* | All nodes matched by *jsonpath2* starting at any node matching *jsonpath1* |
  70. +-------------------------------------+------------------------------------------------------------------------------------+
  71. | *jsonpath* ``[`` *whatever* ``]`` | Same as *jsonpath*\ ``.``\ *whatever* |
  72. +-------------------------------------+------------------------------------------------------------------------------------+
  73. | *jsonpath1* ``..`` *jsonpath2* | All nodes matched by *jsonpath2* that descend from any node matching *jsonpath1* |
  74. +-------------------------------------+------------------------------------------------------------------------------------+
  75. | *jsonpath1* ``where`` *jsonpath2* | Any nodes matching *jsonpath1* with a child matching *jsonpath2* |
  76. +-------------------------------------+------------------------------------------------------------------------------------+
  77. | *jsonpath1* ``|`` *jsonpath2* | Any nodes matching the union of *jsonpath1* and *jsonpath2* |
  78. +-------------------------------------+------------------------------------------------------------------------------------+
  79. Field specifiers ( *field* ):
  80. +-------------------------+-------------------------------------------------------------------------------------+
  81. | Syntax | Meaning |
  82. +=========================+=====================================================================================+
  83. | ``fieldname`` | the field ``fieldname`` (from the "current" object) |
  84. +-------------------------+-------------------------------------------------------------------------------------+
  85. | ``"fieldname"`` | same as above, for allowing special characters in the fieldname |
  86. +-------------------------+-------------------------------------------------------------------------------------+
  87. | ``'fieldname'`` | ditto |
  88. +-------------------------+-------------------------------------------------------------------------------------+
  89. | ``*`` | any field |
  90. +-------------------------+-------------------------------------------------------------------------------------+
  91. | *field* ``,`` *field* | either of the named fields (you can always build equivalent jsonpath using ``|``) |
  92. +-------------------------+-------------------------------------------------------------------------------------+
  93. Array specifiers ( *idx* ):
  94. +-----------------------------------------+---------------------------------------------------------------------------------------+
  95. | Syntax | Meaning |
  96. +=========================================+=======================================================================================+
  97. | ``[``\ *n*\ ``]`` | array index (may be comma-separated list) |
  98. +-----------------------------------------+---------------------------------------------------------------------------------------+
  99. | ``[``\ *start*\ ``?:``\ *end*\ ``?]`` | array slicing (note that *step* is unimplemented only due to lack of need thus far) |
  100. +-----------------------------------------+---------------------------------------------------------------------------------------+
  101. | ``[*]`` | any array index |
  102. +-----------------------------------------+---------------------------------------------------------------------------------------+
  103. Programmatic JSONPath
  104. ---------------------
  105. If you are programming in Python and would like a more robust way to
  106. create JSONPath expressions that does not depend on a parser, it is very
  107. easy to do so directly, and here are some examples:
  108. - ``Root()``
  109. - ``Slice(start=0, end=None, step=None)``
  110. - ``Fields('foo', 'bar')``
  111. - ``Index(42)``
  112. - ``Child(Fields('foo'), Index(42))``
  113. - ``Where(Slice(), Fields('subfield'))``
  114. - ``Descendants(jsonpath, jsonpath)``
  115. Extensions
  116. ----------
  117. - *Path data*: The result of ``JsonPath.find`` provide detailed context
  118. and path data so it is easy to traverse to parent objects, print full
  119. paths to pieces of data, and generate automatic ids.
  120. - *Automatic Ids*: If you set ``jsonpath_rw.auto_id_field`` to a value
  121. other than None, then for any piece of data missing that field, it
  122. will be replaced by the JSONPath to it, giving automatic unique ids
  123. to any piece of data. These ids will take into account any ids
  124. already present as well.
  125. - *Named operators*: Instead of using ``@`` to reference the currently
  126. object, this library uses ```this```. In general, any string
  127. contained in backquotes can be made to be a new operator, currently
  128. by extending the library.
  129. More to explore
  130. ---------------
  131. There are way too many jsonpath implementations out there to discuss.
  132. Some are robust, some are toy projects that still work fine, some are
  133. exercises. There will undoubtedly be many more. This one is made for use
  134. in released, maintained code, and in particular for programmatic access
  135. to the abstract syntax and extension. But JSONPath at its simplest just
  136. isn't that complicated, so you can probably use any of them
  137. successfully. Why not this one?
  138. The original proposal, as far as I know:
  139. - `JSONPath - XPath for
  140. JSON <http://goessner.net/articles/JSONPath/>`__ by Stefan Goessner.
  141. Special note about PLY and docstrings
  142. -------------------------------------
  143. The main parsing toolkit underlying this library,
  144. `PLY <https://github.com/dabeaz/ply>`__, does not work with docstrings
  145. removed. For example, ``PYTHONOPTIMIZE=2`` and ``python -OO`` will both
  146. cause a failure.
  147. Contributors
  148. ------------
  149. This package is authored and maintained by:
  150. - `Kenn Knowles <https://github.com/kennknowles>`__
  151. (`@kennknowles <https://twitter.com/KennKnowles>`__)
  152. with the help of patches submitted by `these contributors <https://github.com/kennknowles/python-jsonpath-rw/graphs/contributors>`__.
  153. Copyright and License
  154. ---------------------
  155. Copyright 2013- Kenneth Knowles
  156. Licensed under the Apache License, Version 2.0 (the "License"); you may
  157. not use this file except in compliance with the License. You may obtain
  158. a copy of the License at
  159. ::
  160. http://www.apache.org/licenses/LICENSE-2.0
  161. Unless required by applicable law or agreed to in writing, software
  162. distributed under the License is distributed on an "AS IS" BASIS,
  163. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  164. See the License for the specific language governing permissions and
  165. limitations under the License.
  166. .. |Build Status| image:: https://travis-ci.org/kennknowles/python-jsonpath-rw.png?branch=master
  167. :target: https://travis-ci.org/kennknowles/python-jsonpath-rw
  168. .. |Test coverage| image:: https://coveralls.io/repos/kennknowles/python-jsonpath-rw/badge.png?branch=master
  169. :target: https://coveralls.io/r/kennknowles/python-jsonpath-rw
  170. .. |PyPi version| image:: https://pypip.in/v/jsonpath-rw/badge.png
  171. :target: https://pypi.python.org/pypi/jsonpath-rw
  172. .. |PyPi downloads| image:: https://pypip.in/d/jsonpath-rw/badge.png
  173. :target: https://pypi.python.org/pypi/jsonpath-rw