jsoup Java HTML Parser release 1.11.1

2017-Nov-4

This one goes to eleven! jsoup 1.11.1 features a 30% lower DOM memory footprint, streaming network HTML parsing, faster HTML generation, and a bunch of other improvements and bug fixes.

jsoup is a Java library for working with real-world HTML. It provides a very convenient API for extracting and manipulating data, using the best of DOM, CSS, and jquery-like methods.

Download jsoup now.

Improvements

  • When loading content from a URL or a file, the content is now parsed as it streams in from the network or disk, rather than being fully buffered before parsing. This substantially reduces memory consumption & large garbage objects when loading large files. Note that this change means that a response, once parsed, may not be parsed again from the same response object unless you call Connection.Response.bufferUp() first, which will buffer the full response into memory.
  • Updated language level to Java 7 from Java 5. To maintain Android support (of minversion 8), try-with-resources are not used.
  • Added Connection.Response.bodyStream(), a method to get the response body as an input stream. This is useful for saving a large response straight to a file, without buffering fully into memory first.
  • Performance improvements in text and HTML generation (through less GC).
  • Reduced memory consumption of text, scripts, and comments in the DOM by 40%, by refactoring the node hierarchy to not track childnodes or attributes by default for lead nodes. For the average document, that's about a 30% memory reduction.
  • Reduced memory consumption of Elements by refactoring their Attributes to be a simple pair of arrays, vs a LinkedHashSet.
  • Added support for Element.selectFirst(), to efficiently find the first matching element.
  • Added Element.appendTo(parent) to simplify slinging elements about.
  • Added support for multiple headers with the same name in Jsoup.Connect
  • Added Element.shallowClone() and Node.shallowClone(), to allow cloning nodes without getting all their children.
  • Updated Element.text() and the :contains(text) selector to consider   character as spaces.
  • Updated Jsoup.connect().timeout() to implement a total connect + combined read timeout. Previously it specified connect and buffer read times only, so to implement a combined total timeout, you had to have another thread send an interupt.
  • Improved performance of Node.addChildren() (was quadratic)
  • Added missing support for template tags in tables
  • In Jsoup.Connect file uploads, added the ability to set the uploaded files' mimetype.
  • Improved Node traversal, including less object creation, and partial and filtering traversor support.

Bug Fixes

  • Bugfix: if a document was was redecoded after character set detection, the HTML parser was not reset correctly, which could lead to an incorrect DOM.
  • Bugfix: attributes with the same name but different case would be incorrectly treated as different attributes.
  • Bugfix: self-closing tags for known empty elements were incorrectly treated as errors.
  • Bugfix: fixed an issue where a self-closing title, noframes, or style tag would cause the rest of the page to be incorrectly parsed as data or text.
  • Bugfix: fixed an issue with unknown mixed-case tags
  • Bugfix: fixed an issue where the entity resources were left open after startup, causing a warning.
  • Bugfix: fixed an issue where Element.getElementsByIndexLessThan(index) would incorrectly provide the root element
  • Improved parse time for pages with exceptionally deeply nested tags.
  • Improvement / workaround: modified the Entities implementation to load its data from a .class vs from a jar resource. Faster, and safer on Android.

Many thanks to everyone who contributed patches, suggestions, and bug reports. If you have any suggestions for the next release, I would love to hear them; please get in touch via the mailing list or to me directly.