jsoup

jsoup Java HTML Parser release 1.23.2

2026-08-26T00:00:00Z

jsoup 1.23.2 is out now. This release focuses largely on bug fixes, specification correctness, and performance improvements.

It brings closer alignment with the HTML, XML, URL, and form submission specifications; improves XML and W3C DOM conversion; and makes HTTP workloads more efficient through streamed request bodies and broader JDK HttpClient reuse.

This version also includes new node insertion methods for Elements, and DOM mutations now reject operations that would create a cycle.

jsoup is a Java library for working with real-world HTML and XML. It provides a very convenient API for extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.

Download jsoup now.

Improvements

Improved consecutive StreamParser.selectFirst() calls during progressive parsing, so later matches are returned with their parsed contents when earlier selections had left them as parser lookahead. E.g., given <title>One</title><p id=hit>Full</p><p>Next</p>, selecting title and then #hit now advances the partial lookahead and returns <p id="hit">Full</p>, rather than returning an empty <p id="hit"></p> before its content is parsed. The updated readiness tracking follows StreamParser’s normal emission order across implicit HTML structure and parser recovery. #2551
Improved XML parser performance and memory use for documents with many nested namespace declarations by recording namespace changes within each element scope. #2556
Improved W3CDom conversion performance for documents with many nested namespace declarations. The W3C converter now uses the same optimized namespace tracking as the XML parser. #2559
Improved W3CDom XML conversion to retain processing instructions, comments outside the root element, and CDATA sections, which were previously dropped or converted to text. #2572
DOM mutation methods, including child insertion and replacement, now reject operations that would create a cycle, such as making a node its own child or moving an ancestor beneath a descendant. #2552
Added Elements#before(Node), after(Node), prepend(Node), and append(Node) to match the existing HTML string methods. #953
Large file-backed uploads through Connection.requestBodyStream(InputStream) now stream directly with the JDK HttpClient on Java 11+, rather than being loaded fully into memory first. #2575
Extended Java 11+ HTTP client reuse from requests sharing a Jsoup.newSession() to ordinary Jsoup.connect() calls, reducing transport thread and connection setup churn under sustained request loads. Sessions with custom authentication or SSL contexts continue to use their own client. #2584

Changes

Aligned the XML parser stack depth and lookups to the configured maximum, which now defaults to 512 for both HTML and XML. Use Parser#setMaxDepth(int) to configure. #2570

Bug Fixes

Fixed W3CDom namespace conversion in several cases: #2559
- Namespace declarations and prefixed attributes now carry the correct namespace URI, so namespace-aware DOM lookups work as expected.
- Attributes added after parsing, or included through subtree conversion, now use inherited prefix declarations.
- Namespace declarations now apply regardless of attribute order, and an empty declaration shadows an inherited binding only within its scope.
- With namespace awareness disabled, inherited and undeclared prefixes now receive the declarations needed for XML serialization.
- Valid HTML names that are not XML QNames, such as a:b:c, are normalized. Attributes that still cannot be represented are skipped, and unrepresentable elements no longer change the surrounding tree.
Fixed W3CDom conversion of programmatically created or renamed elements whose names can be represented in a jsoup HTML DOM but are not valid XML names, such as 1abc. These names are now normalized (e.g. _1abc) instead of causing a NullPointerException. #2560
Fixed XML doctype serialization when a system identifier contains a double quote, which could otherwise produce invalid XML. #2571
XML serialization now repairs element and attribute names that start with an invalid character, rather than outputting null elements or dropping attributes. For example, an attribute named 1a is written as _1a. Additional leading underscores keep repaired attribute names unique if they conflict with another attribute. #2573
Supplementary Unicode characters are now escaped correctly when serializing with non-UTF, non-ASCII output charsets such as ISO-8859-1. Previously, characters could be emitted unescaped when their low 16-bit value was representable by the configured charset, causing replacement or corruption when the output was encoded. #2578
Fixed the JDK HttpClient implementation to accept responses missing a Content-Type header, matching the HttpURLConnection implementation. #2549
Fixed HTTP response content-type matching to handle media types case-insensitively and recognize structured +xml suffixes, including vendor-specific media types. #2550
HTTP request URL normalization now percent-encodes ASCII control characters, DEL, and embedded fragment delimiters, keeping normalized URLs valid for HTTP requests while preserving existing escapes. #2585
Corrected multipart form encoding to percent-escape CR and LF in field names and filenames, matching the HTML form submission specification. Multipart file content-types containing CR or LF are now rejected with a ValidationException. #2555
Aligned trailing comment placement with the HTML specification: comments after </body> remain children of the html element, while comments after </html> remain children of the document. #2557
When using the optional re2j regular expression engine, memory allocation errors caused by complex selector patterns at match time are now normalized to a ValidationException with a Pattern complexity error message.
Fixed parsing of malformed SVG and MathML content so that breakout HTML tags are placed according to the HTML specification. #2562
Fixed deeply nested malformed HTML parsing that could lose the document body because stack lookups did not align to the configured maximum parser depth. #2569
Aligned RCDATA, RAWTEXT, and script-data parsing with the HTML specification: malformed end tags no longer consume following markup, unclosed title/textarea content stays text through EOF, and custom text tags match exact names. #2577
Improved URL validation during HTTP/HTTPS URL resolution and cleaning; resolved URLs without a host are now rejected instead of being accepted based only on their scheme prefix, aligning to RFC 9110. Valid relative links and non-HTTP(S) schemes are unchanged. #2579
Redirects with malformed single-slash HTTP locations now use standard URL resolution to align with browsers. #2580
Template fragment parsing now handles unmatched </template> tags without throwing a ValidationException. #2581
Improved source tracking for adopted formatting elements and malformed markup ending at EOF. #2582

My sincere thanks to everyone who contributed to this release! If you have any suggestions for the next release, I would love to hear them; please get in touch via jsoup discussions, or with me directly.

You can also follow me (@jhy@tilde.zone) on Mastodon (Fediverse) to receive occasional notes about jsoup releases.

jsoup Java HTML Parser release 1.23.1

2026-07-30T00:00:00Z

jsoup 1.23.1 is out now, with a faster and more memory-efficient HTML parser, improved alignment with the HTML standard across noscript, CDATA, SVG, and MathML parsing, and safer, specification-correct HTTP redirects. The release also adds an immutable Element#classList(), direct outer-HTML output to an Appendable, and fixes across RCDATA parsing, XML conversion, tag-name handling, and Cleaner link detection.

Performance optimization was a major focus for this release. In our OpenJDK 21 benchmarks, ordinary string parsing is now 18% faster on average, parsing from an InputStream is 11% faster, and parsing with source position tracking is 70% faster while allocating 64% fewer bytes per document.

Source-tracked DOMs retain 58-65% less memory on representative medium-to-large documents, and the improvements hold under concurrent parsing without introducing new contention. Exact gains will naturally vary with document shape, JVM, and hardware.

This release also fixes a security issue in the Cleaner that could expose markup when malformed HTML is cleaned with a custom Safelist permitting certain raw-text elements. The built-in Safelists are unaffected.

jsoup is a Java library for working with real-world HTML and XML. It provides a very convenient API for extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.

Download jsoup now.

Improvements

Reduced retained memory when parsing with source position tracking enabled (Parser#setTrackPosition(true)). Source ranges are now stored in compact parser-owned span records instead of node and attribute user data, and Position objects are created lazily when source ranges are read. This cuts tracked DOM retained size by about 50-60% on representative benchmark documents, while keeping Node#sourceRange(), Element#endSourceRange(), and Attribute#sourceRange() behavior intact. #2498
Added Element#classList(), an immutable snapshot of an element’s class names in attribute order. Use hasClass() when you just need to test for one class, classList() when you want to read or iterate classes without needing a mutable result, and classNames() when you want the existing mutable, deduplicated set that can be written back with classNames(Set). The class APIs now share an HTML-whitespace scanner, which also makes classNames() faster and lighter on allocation, especially when walking many elements without class names. #2500
Aligned HTML parser scope classification with the current HTML spec for select, foreignObject, and template. #2501
Simplified the HTML tree builder’s scope, implied-end-tag, and special-element checks by caching parser-only options on Tag. That improves HTML parser throughput by about 10% on small inputs and up to about 30% on larger inputs in the benchmark fixtures. #2502
Improved HTML parser throughput stability by making hot tokeniser scan paths compile more predictably. #2507
<noscript> fallback markup is now parsed into an inspectable DOM subtree in both the document head and body. The fallback acts as a contained parsing island, so malformed markup cannot disrupt the surrounding document structure, while normal HTML tokenization still applies within it. This also improves round-trip serialization. #2537
Improved redirect credential handling as a defense-in-depth measure: explicit authorization headers and request cookies are no longer forwarded across origins, reducing exposure through open redirects and aligning with HTTP guidance. Cookies managed by a CookieStore continue to follow their configured scope. #2540
Elements can now append their outer HTML, including their own tags, directly to an Appendable with Node#outerHtml(Appendable), without first creating a String. This complements Element#html(Appendable), which appends inner HTML only. #2532
Aligned CDATA tokenization with the HTML spec: CDATA syntax in HTML content is parsed as a bogus comment, while it remains supported in SVG, MathML, and XML. Also improved namespace-aware fragment parsing so SVG and MathML contexts, HTML integration points, and context-sensitive tokenizer states are handled correctly. #2542
When using the optional re2j regular expression engine, stack overflows caused by complex selector patterns are now normalized to a ValidationException with a Pattern complexity error message. #2548

Bug Fixes

Fixed HTML parsing of mixed-case RCDATA end tags after tag-shaped text. For example, <title><p>Foo</TiTLE> and <textarea><img src=x></TeXtArEa> now keep the tag-shaped content as text instead of promoting it to markup. #2503
Fixed W3CDom XML conversion so plain XML elements don’t serialize with the reserved XML namespace as the default namespace. Explicit XML namespaces and xml:* attributes are still preserved. #2504
Preserve control characters in parsed tag names #2538
Updated HTTP redirects to follow the specification: 307 and 308 preserve the request method and content, 301 and 302 only change POST to GET, and Location is followed only for 301, 302, 303, 307, and 308 responses. Streamed request bodies are not buffered; if an automatic redirect requires replaying one, execution fails, so the caller can resend with a fresh stream. #2540
Corrected the Cleaner’s same-site link detection to compare hostnames rather than URL prefixes when applying rel=nofollow. #2543

Build Changes

Cleaned up the Maven build for the multi-release JAR so Java 8 and Java 11+ sources compile as separate source sets. This avoids spurious Java 8 compiler warnings from newer-language overlay sources, keeps long-running parser checks behind an explicit profile, and preserves the same published artifacts and runtime behavior.
Improved parallelism and tuned timing in our integration tests, so that a full mvn clean verify drops from ~ 1m18s to ~ 21 seconds.

You can also follow me (@jhy@tilde.zone) on Mastodon / Fediverse to receive occasional notes about jsoup releases.

jsoup Java HTML Parser release 1.22.2

2026-04-20T00:00:00Z

jsoup 1.22.2 is out now, with fixes and refinements across the library. It makes editing the DOM during traversal more predictable, refreshes the default HTML tag definitions with newer elements and better text boundaries, and improves reliability in parsing and HTTP transport. The release also fixes a number of edge cases in cleaning, stream parsing, XML doctype handling, and Android packaging.

jsoup is a Java library for working with real-world HTML and XML. It provides a very convenient API for extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.

Download jsoup now.

Improvements

Expanded and clarified NodeTraversor support for in-place DOM rewrites during NodeVisitor.head(). Current-node edits such as remove, replace, and unwrap now recover more predictably, while traversal stays within the original root subtree. This makes single-pass tree cleanup and normalization visitors easier to write, for example when unwrapping presentational elements or replacing text nodes as you walk the DOM. #2472
Documentation: clarified that a configured Cleaner may be reused across concurrent threads, and that shared Safelist instances should not be mutated while in use. #2473
Updated the default HTML TagSet for current HTML elements: added dialog, search, picture, and slot; made ins, del, button, audio, video, and canvas inline by default (Tag#isInline(), aligned to phrasing content in the spec); and added readable Element.text() boundaries for controls and embedded objects via the new Tag.TextBoundary option. This improves pretty-printing and keeps normalized text from running adjacent words together. #2493

Bug Fixes

Android (R8/ProGuard): added a rule to ignore the optional re2j dependency when not present. #2459
Fixed a NodeTraversor regression in 1.21.2 where removing or replacing the current node during head() could revisit the replacement node and loop indefinitely. The traversal docs now also clarify which inserted nodes are visited in the current pass. #2472
Parsing during charset sniffing no longer fails if an advisory available() call throws IOException, as seen on JDK 8 HttpURLConnection. #2474
Cleaner no longer makes relative URL attributes in the input document absolute when cleaning or validating a Document. URL normalization now applies only to the cleaned output, and Safelist.isSafeAttribute() is side effect free. #2475
Cleaner no longer duplicates enforced attributes when the input Document preserves attribute case. A case-variant source attribute is now replaced by the enforced attribute in the cleaned output. #2476
If a per-request SOCKS proxy is configured, jsoup now avoids using the JDK HttpClient, because the JDK would silently ignore that proxy and attempt to connect directly. Those requests now fall back to the legacy HttpURLConnection transport instead, which does support SOCKS. #2468
Connection.Response.streamParser() and DataUtil.streamParser(Path, ...) could fail on small inputs without a declared charset, if the initial 5 KB charset sniff fully consumed the input and closed it before the stream parse began. #2483
In XML mode, doctypes with an internal subset, such as <!DOCTYPE root [<!ENTITY name "value">]>, now round-trip correctly. The subset is preserved as raw text only; entities are not expanded and external DTDs are not loaded. #2486

Build Changes

Migrated the integration test server from Jetty to Netty, which actively maintains support for our minimum JDK target (8). #2491

You can also follow me (@jhy@tilde.zone) on Mastodon / Fediverse to receive occasional notes about jsoup releases.

jsoup Java HTML Parser release 1.22.1

2026-01-01T00:00:00Z

jsoup 1.22.1 is out now, adding support for the re2j regular expression engine for regex-based CSS selectors, a configurable maximum parser depth, and numerous bug fixes and improvements.

jsoup is a Java library for working with real-world HTML and XML. It provides a very convenient API for extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.

Download jsoup now.

Improvements

Added support for using the re2j regular expression engine for regex-based CSS selectors (e.g. [attr~=regex], :matches(regex)), which ensures linear-time performance for regex evaluation. This allows safer handling of arbitrary user-supplied query regexes. To enable, add the com.google.re2j dependency to your classpath, e.g.:

  <dependency>
    <groupId>com.google.re2j</groupId>
    <artifactId>re2j</artifactId>
    <version>1.8</version>
  </dependency>

(If you already have that dependency in your classpath, but you want to keep using the Java regex engine, you can disable re2j via System.setProperty("jsoup.useRe2j", "false").) You can confirm that the re2j engine has been enabled correctly by calling Regex.usingRe2j(). #2407

Added an instance method Parser#unescape(String, boolean) that unescapes HTML entities using the parser’s configuration (e.g. to support error tracking), complementing the existing static utility Parser.unescapeEntities(String, boolean). #2396
Added a configurable maximum parser depth (to limit the number of open elements on stack) to both HTML and XML parsers. The HTML parser now defaults to a depth of 512 to match browser behavior, and protect against unbounded stack growth, while the XML parser keeps unlimited depth by default, but can opt into a limit via Parser.setMaxDepth(). #2421
Build: added CI coverage for JDK 25 #2403
Build: added a CI fuzzer for contextual fragment parsing (in addition to existing full body HTML and XML fuzzers). oss-fuzz #14041

Changes

Set a removal schedule of jsoup 1.24.1 for previously deprecated APIs.

Bug Fixes

Previously cached child Elements of an Element were not correctly invalidated in Node#replaceWith(Node), which could lead to incorrect results when subsequently calling Element#children(). #2391
Attribute selector values are now compared literally without trimming. Previously, jsoup trimmed whitespace from selector values and from element attribute values, which could cause mismatches with browser behavior (e.g. [attr=" foo "]). Now matches align with the CSS specification and browser engines. #2380
When using the JDK HttpClient, any system default proxy (ProxySelector.getDefault()) was ignored. Now, the system proxy is used if a per-request proxy is not set. #2388, #2390
A ValidationException could be thrown in the adoption agency algorithm with particularly broken input. Now logged as a parse error. #2393
Null characters in the HTML body were not consistently removed; and in foreign content were not correctly replaced. #2395
An IndexOutOfBoundsException could be thrown when parsing a body fragment with crafted input. Now logged as a parse error. #2397, #2406
When using StructuralEvaluators (e.g., a parent child selector) across many retained threads, their memoized results could also be retained, increasing memory use. These results are now cleared immediately after use, reducing overall memory consumption. #2411
Cloning a Parser now preserves any custom TagSet applied to the parser. #2422, #2423
Custom tags marked as Tag.Void now parse and serialize like the built-in void elements: they no longer consume following content, and the XML serializer emits the expected self-closing form. #2425
The <br> element is once again classified as an inline tag (Tag.isBlock() == false), matching common developer expectations and its role as phrasing content in HTML, while pretty-printing and text extraction continue to treat it as a line break in the rendered output. #2387, #2439
Fixed an intermittent truncation issue when fetching and parsing remote documents via Jsoup.connect(url).get(). On responses without a charset header, the initial charset sniff could sometimes (depending on buffering / available() behavior) be mistaken for end-of-stream and a partial parse reused, dropping trailing content. #2448
TagSet copies no longer mutate their template during lazy lookups, preventing cross-thread ConcurrentModificationException when parsing with shared sessions. #2453
Fixed parsing of <svg> foreignObject content nested within a <p>, which could incorrectly move the HTML subtree outside the SVG. #2452

Internal Changes

Deprecated internal helper org.jsoup.internal.Functions (for removal in v1.23.1). This was previously used to support older Android API levels without full java.util.function coverage; jsoup now requires core library desugaring so this indirection is no longer necessary. #2412

You can also follow me (@jhy@tilde.zone) on Mastodon / Fediverse to receive occasional notes about jsoup releases.

jsoup Java HTML Parser release 1.21.2

2025-08-25T00:00:00Z

jsoup 1.21.2 is out now, adding support for custom SSLContext in HTTP/2 connections, and improving consistency in how user data is handled in attributes. It also brings performance gains in DOM manipulation and fragment parsing, and fixes several edge cases in stream parsing, traversal, cloning, and concurrent reads.

jsoup is a Java library for working with real-world HTML and XML. It provides a very convenient API for extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.

Changes

Deprecated internal (yet visible) methods Normalizer#normalize(String, bool) and Attribute#shouldCollapseAttribute(Document.OutputSettings). These will be removed in a future version.
Deprecated Connection#sslSocketFactory(SSLSocketFactory) in favor of the new Connection#sslContext(SSLContext). Using sslSocketFactory will force the use of the legacy HttpUrlConnection implementation, which does not support HTTP/2. #2370

Improvements

When pretty-printing, if there are consecutive text nodes (via DOM manipulation), the non-significant whitespace between them will be collapsed. #2349.
Updated Connection.Response#statusMessage() to return a simple loggable string message (e.g. “OK”) when using the HttpClient implementation, which doesn’t otherwise return any server-set status message. #2356
Attributes#size() and Attributes#isEmpty() now exclude any internal attributes (such as user data) from their count. This aligns with the attributes’ serialized output and iterator. #2369
Added Connection#sslContext(SSLContext) to provide a custom SSL (TLS) context to requests, supporting both the HttpClient and the legacy HttUrlConnection implementations. #2370
Performance optimizations for DOM manipulation methods including when repeatedly removing an element’s first child (element.child(0).remove()), and when using Parser#parseBodyFragement() to parse a large number of direct children. #2373.

Bug Fixes

When parsing from an InputStream and a multibyte character happened to straddle a buffer boundary, the stream would not be completely read. #2353.
In NodeTraversor, if a last child element was removed during the head() call, the parent would be visited twice. #2355.
Cloning an Element that has an Attributes object would add an empty internal user-data attribute to that clone, which would cause unexpected results for Attributes#size() and Attributes#isEmpty(). #2356
In a multithreaded application where multiple threads are calling Element#children() on the same element concurrently, a race condition could happen when the method was generating the internal child element cache (a filtered view of its child nodes). Since concurrent reads of DOM objects should be threadsafe without external synchronization, this method has been updated to execute atomically. #2366
When parsing HTML with svg:script elements in SVG elements, don’t enter the Text insertion mode, but continue to parse as foreign content. Otherwise, misnested HTML could then cause an IndexOutOfBoundsException. #2374
Malformed HTML could throw an IndexOutOfBoundsException during the adoption agency. #2377

You can also follow me (@jhy@tilde.zone) on Mastodon / Fediverse to receive occasional notes about jsoup releases.

Download jsoup now.

jsoup Java HTML Parser release 1.21.1

2025-06-23T00:00:00Z

jsoup 1.21.1 is out now, featuring powerful new node selection capabilities that let you target specific DOM nodes like comments and text nodes using CSS selectors, dynamic tag customization through the new TagSet callback system, and improved defense against mutation XSS attacks with simplified attribute escaping. This release also brings HTTP/2 support by default, numerous API improvements for better developer experience, and fixes for several edge-case parsing issues.

jsoup is a Java library for working with real-world HTML and XML. It provides a very convenient API for extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.

Changes

Removed previously deprecated methods. #2317
Deprecated the :matchText pseduo-selector due to its side effects on the DOM; use the new ::textnode selector and the Element#selectNodes(String css, Class<T> type) method instead. #2343
Deprecated Connection.Response#bufferUp() in lieu of Connection.Response#readFully() which can throw a checked IOException.
Deprecated internal methods Validate#ensureNotNull(Object) (replaced by typed Validate#expectNotNull(T)); protected HTML appenders from Attribute and Node.
If you happen to be using any of the deprecated methods, please take the opportunity now to migrate away from them, as they will be removed in a future release.

Improvements

Enhanced the Selector to support direct matching against nodes such as comments and text nodes. For example, you can now find an element that follows a specific comment: ::comment:contains(prices) + p will select p elements immediately after a  comment. Supported types include ::node, ::leafnode, ::comment, ::text, ::data, and ::cdata. Node contextual selectors like ::node:contains(text), :matches(regex), and :blank are also supported. Introduced Element#selectNodes(String css) and Element#selectNodes(String css, Class<T> nodeType) for direct node selection. #2324
Added TagSet#onNewTag(Consumer<Tag> customizer): register a callback that’s invoked for each new or cloned Tag when it’s inserted into the set. Enables dynamic tweaks of tag options (for example, marking all custom tags as self-closing, or everything in a given namespace as preserving whitespace). #2330
Made TokenQueue and CharacterReader autocloseable, to ensure that they will release their buffers back to the buffer pool, for later reuse.
Added Selector#evaluatorOf(String css), as a clearer way to obtain an Evaluator from a CSS query. An alias of QueryParser.parse(String css).
Custom tags (defined via the TagSet) in a foreign namespace (e.g. SVG) can be configured to parse as data tags.
Added NodeVisitor#traverse(Node) to simplify node traversal calls (vs. importing NodeTraversor).
Updated the default user-agent string to improve compatibility. #2341
The HTML parser now allows the specific text-data type (Data, RcData) to be customized for known tags. (Previously, that was only supported on custom tags.) #2326
Added Connection.Response#readFully() as a replacement for Connection.Response#bufferUp() with an explicit IOException. Similarly, added Connection.Response#readBody() over Connection.Response#body(). Deprecated Connection.Response#bufferUp(). #2327
When serializing HTML, the < and > characters are now escaped in attributes. This helps prevent a class of mutation XSS attacks. #2337
Changed Connection to prefer using the JDK’s HttpClient over HttpUrlConnection, if available, to enable HTTP/2 support by default. Users can disable via -Djsoup.useHttpClient=false. #2340

Bug Fixes

The contents of a script in a svg foreign context should be parsed as script data, not text. #2320
Tag#isFormSubmittable() was updating the Tag’s options. #2323
The HTML pretty-printer would incorrectly trim whitespace when text followed an inline element in a block element. #2325
Custom tags with hyphens or other non-letter characters in their names now work correctly as Data or RcData tags. Their closing tags are now tokenized properly. #2332
When cloning an Element, the clone would retain the source’s cached child Element list (if any), which could lead to incorrect results when modifying the clone’s child elements. #2334

You can also follow me (@jhy@tilde.zone) on Mastodon / Fediverse to receive occasional notes about jsoup releases.

Downloadjsoup now.

jsoup Java HTML Parser release 1.20.1

2025-04-29T00:00:00Z

jsoup 1.20.1 is out now, featuring improved HTML parse rules to align with modern browsers, improved XML namespace handling, and a redesigned HTML pretty-printer for better consistency and customizability. This release also delivers performance optimizations, new API enhancements such as flexible tag definitions via TagSet, concise CSS selectors, and parser thread-safety improvements. Additionally, multiple bug fixes enhance XML serialization and W3C DOM interoperability.

jsoup is a Java library for working with real-world HTML and XML. It provides a very convenient API for extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.

Changes

To better follow the HTML5 spec and current browsers, the HTML parser no longer allows self-closing tags (<foo />) to close HTML elements by default. Foreign content (SVG, MathML), and content parsed with the XML parser, still supports self-closing tags. If you need specific HTML tags to support self-closing, you can register a custom tag via the TagSet configured in Parser.tagSet(), using Tag#set(Tag.SelfClose). Standard void tags (such as <img>, <br>, etc.) continue to behave as usual and are not affected by this change. #2300.
The following internal components have been deprecated. If you do happen to be using any of these, please take the opportunity now to migrate away from them, as they will be removed in jsoup 1.21.1.
- ChangeNotifyingArrayList, Document.updateMetaCharsetElement(), Document.updateMetaCharsetElement(boolean), HtmlTreeBuilder.isContentForTagData(String), Parser.isContentForTagData(String), Parser.setTreeBuilder(TreeBuilder), Tag.formatAsBlock(), Tag.isFormListed(), TokenQueue.addFirst(String), TokenQueue.chompTo(String), TokenQueue.chompToIgnoreCase(String), TokenQueue.consumeToIgnoreCase(String), TokenQueue.consumeWord(), TokenQueue.matchesAny(String...)

Functional Improvements

Rebuilt the HTML pretty-printer, to simplify and consolidate the implementation, improve consistency, support custom Tags, and provide a cleaner path for ongoing improvements. The specific HTML produced by the pretty-printer may be different from previous versions. #2286.
Added the ability to define custom tags, and to modify properties of known tags, via the TagSet tag collection. Their properties can impact both the parse and how content is serialized (output as HTML or XML). #2285.
Element.cssSelector() will prefer to return shorter selectors by using ancestor IDs when available and unique. E.g. #id > div > p instead of html > body > div > div > p #2283.
Added Elements.deselect(int index), Elements.deselect(Object o), and Elements.deselectAll() methods to remove elements from the Elements list without removing them from the underlying DOM. Also added Elements.asList() method to get a modifiable list of elements without affecting the DOM. (Individual Elements remain linked to the DOM.) #2100.
Added support for sending a request body from an InputStream with Connection.requestBodyStream(InputStream stream). #1122.
The XML parser now supports scoped xmlns: prefix namespace declarations, and applies the correct namespace to Tags and Attributes. Also, added Tag#prefix(), Tag#localName(), Attribute#prefix(), Attribute#localName(), and Attribute#namespace() to retrieve these. #2299.
CSS identifiers are now escaped and unescaped correctly to the CSS spec. Element#cssSelector() will emit appropriately escaped selectors, and the QueryParser supports those. Added Selector.escapeCssIdentifier() and ` Selector.unescapeCssIdentifier(). #2297, #2305

Structure and Performance Improvements

Refactored the CSS QueryParser into a clearer recursive descent parser. #2310.
CSS selectors with consecutive combinators (e.g. div >> p) will throw an explicit parse exception. #2311.
Performance: reduced the shallow size of an Element from 40 to 32 bytes, and the NodeList from 32 to 24. #2307.
Performance: reduced GC load of new StringBuilders when tokenizing input HTML. #2304.
Made Parser instances threadsafe, so that inadvertent use of the same instance across threads will not lead to errors. For actual concurrency, use Parser#newInstance() per thread. #2314.

Bug Fixes

Element names containing characters invalid in XML are now normalized to valid XML names when serializing. #1496.
When serializing to XML, characters that are invalid in XML 1.0 should be removed (not encoded). #1743.
When converting a Document to the W3C DOM in W3CDom, elements with an attribute in an undeclared namespace now get a declaration of xmlns:prefix="undefined". This allows subsequent serialization to XML via W3CDom.asString() to succeed. #2087.
The StreamParser could emit the final elements of a document twice, due to how onNodeCompleted was fired when closing out the stack. #2295.
When parsing with the XML parser and error tracking enabled, the trailing ? in <?xml version="1.0"?> would incorrectly emit an error. #2298.
Calling Element#cssSelector() on an element with combining characters in the class or ID now produces the correct output. #1984.

You can also follow me (@jhy@tilde.zone) on Mastodon / Fediverse to receive occasional notes about jsoup releases.

Download jsoup now.

jsoup Java HTML Parser release 1.19.1

2025-03-04T00:00:00Z

jsoup 1.19.1 is out now, with support for http/2 network requests, performance improvements, some new API methods, and a host of other improvements and bug fixes.

jsoup is a Java library for working with real-world HTML and XML. It provides a very convenient API for extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.

Changes

Added support for http/2 requests in Jsoup.connect(), when running on Java 11+, via the Java HttpClient implementation. #2257.
- In this version of jsoup, the default is to make requests via the HttpUrlConnection implementation: use System.setProperty("jsoup.useHttpClient", "true"); to enable making requests via the HttpClient (if available), which will enable http/2 support. This will become the default in a later version of jsoup, so now is a good time to validate it.
- If you are repackaging the jsoup jar in your deployment (i.e. creating a shaded- or a fat-jar), make sure to specify that as a Multi-Release JAR.
- If the HttpClient impl is not available in your JRE, requests will continue to be made via HttpURLConnection (in http/1.1 mode).
Updated the minimum Android API Level validation from 10 to 21. As with previous jsoup versions, Android developers need to enable core library desugaring. The minimum Java version remains Java 8. #2173
Removed previously deprecated class: org.jsoup.UncheckedIOException (replace with java.io.UncheckedIOException); moved previously deprecated method Element Element#forEach(Consumer) to void Element#forEach(Consumer()). #2246
Deprecated the methods Document#updateMetaCharsetElement(boolean) and Document#updateMetaCharsetElement(), as the setting had no effect. When Document#charset(Charset) is called, the document’s meta charset or XML encoding instruction is always set. #2247

Improvements

When cleaning HTML with a Safelist that preserves relative links, the isValid() method will now consider these links valid. Additionally, the enforced attribute rel=nofollow will only be added to external links when configured in the safelist. #2245
Added Element#selectStream(String query) and Element#selectStream(Evaluator) methods, that return a Stream of matching elements. Elements are evaluated and returned as they are found, and the stream can be terminated early. #2092
Element objects now implement Iterable, enabling them to be used in enhanced for loops.
Added support for fragment parsing from a Reader via Parser#parseFragmentInput(Reader, Element, String). #1177
Reintroduced CLI executable examples, in jsoup-examples.jar. #1702
Optimized performance of selectors like #id .class (and other similar descendant queries) by around 4.6x, by better balancing the Ancestor evaluator’s cost function in the query planner. #2254
Removed the legacy parsing rules for <isindex> tags, which would autovivify a form element with labels. This is no longer in the spec.
Added Elements.selectFirst(String cssQuery) and Elements.expectFirst(String cssQuery), to select the first matching element from an Elements list. #2263
When parsing with the XML parser, XML Declarations and Processing Instructions are directly handled, vs bouncing through the HTML parser’s bogus comment handler. Serialization for non-doctype declarations no longer end with a spurious !. #2275
When converting parsed HTML to XML or the W3C DOM, element names containing < are normalized to _ to ensure valid XML. For example, <foo<bar> becomes <foo_bar>, as XML does not allow < in element names, but HTML5 does. #2276
Reimplemented the HTML5 Adoption Agency Algorithm to the current spec. This handles mis-nested formating / structural elements. #2278

Bug Fixes

If an element has an ; in an attribute name, it could not be converted to a W3C DOM element, and so subsequent XPath queries could miss that element. Now, the attribute name is more completely normalized. #2244
For backwards compatibility, reverted the internal attribute key for doctype names to “name”. #2241
In Connection, skip cookies that have no name, rather than throwing a validation exception. #2242
When running on JDK 1.8, the error java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer; could be thrown when calling Response#body() after parsing from a URL and the buffer size was exceeded. #2250
For backwards compatibility, allow null InputStream inputs to Jsoup.parse(InputStream stream, ...), by returning an empty Document. #2252
A template tag containing an li within an open li would be parsed incorrectly, as it was not recognized as a “special” tag (which have additional processing rules). Also, added the SVG and MathML namespace tags to the list of special tags. #2258
A template tag containing a button within an open button would be parsed incorrectly, as the “in button scope” check was not aware of the template element. Corrected other instances including MathML and SVG elements, also. #2271
An :nth-child selector with a negative digit-less step, such as :nth-child(-n+2), would be parsed incorrectly as a positive step, and so would not match as expected. #1147
Calling doc.charset(charset) on an empty XML document would throw an IndexOutOfBoundsException. #2266
Fixed a memory leak when reusing a nested StructuralEvaluator (e.g., a selector ancestor chain like A B C) by ensuring cache reset calls cascade to inner members. #2277
Concurrent calls to doc.clone().append(html) were not supported. When a document was cloned, its Parser was not cloned but was a shallow copy of the original parser. #2281

You can also follow me (@jhy@tilde.zone) on Mastodon / Fediverse to receive occasional notes about jsoup releases.

Download jsoup now.

Parse large documents efficiently with StreamParser

2025-03-02T00:00:00Z

Problem

You need to parse an HTML or XML document that is too large to fit entirely into memory, or you want to process elements progressively as they are encountered. A typical use case is extracting specific elements from a large document, or handling streamed HTML from a network source efficiently.

Traditional Java SAX parsers offer efficient streaming parsing for XML and HTML, but they lack an ergonomic way to traverse or manipulate elements like a DOM parser. Meanwhile, standard DOM parsers, such as Jsoup.parse(), require loading the entire document into memory, which may be inefficient for large files.

Solution

Use the StreamParser, which allows you to parse parsing an HTML or XML document in an event driven hybrid DOM + SAX style. Elements are emitted as they are completed, enabling efficient memory use and incremental processing. This hybrid approach allows you to process elements as they arrive, including their children and ancestors, while still leveraging jsoup’s intuitive API.

This makes StreamParser a viable alternative to traditional SAX parsers while providing a more ergonomic and familiar API. And jsoup’s robust handling of malformed HTML and XML ensures that real-world documents can be processed effectively.

try (StreamParser streamer = Jsoup.connect("https://example.com/large.html")
    .execute()
    .streamParser()) {
    
    Element el;
    while ((el = streamer.selectNext("article")) != null) {
        // Will include the children of <article>
        System.out.println("Processing article: " + el.text());
        el.remove(); // Keep memory usage low by discarding processed elements
    }
}

Description

Unlike the default jsoup parse method, which constructs a full DOM tree in memory, StreamParser allows for progressive parsing:

Elements are fully formed and emitted as they are completed.
The parser can run in an iterator-like fashion with selectNext(query) to fetch elements as needed.
The DOM tree can be pruned during parsing to save memory.
The document() method provides access to the partially built document.
Parsing can be stopped early with stop() if only a portion of the document is needed.
The backing input (a URL connection, or a file) is read incrementally as the parse proceeds, reducing buffer bloat.

A StreamParser can be reused via a new parse(Reader, String), but is not thread-safe for concurrent inputs. New parsers should be used in each thread.

If created via Connection.Response#streamParser(), or another Reader that is I/O backed, the iterator and stream consumers will throw an java.io.UncheckedIOException if the underlying Reader errors during read.

The StreamParser wraps an underlying HTML or XML parser, so the same configuration options can be used as with the standard Jsoup.parse method.

Examples

Process a file in chunks

Let’s say we have an XML file with a bunch of <book> chunks, each with many <chapter> elements, and loading it all into the DOM at once might run out of memory. Parse the file incrementally using DataUtil.streamParser(...). Then process the file in chunks by iterating on selectNext(cssquery):

static void streamChunks(Path path) throws IOException {
try (StreamParser streamer = DataUtil.streamParser(
    path, StandardCharsets.UTF_8, "https://example.com", Parser.xmlParser())) {

    Element el;
    var seenChunks = 0;
    while ((el = streamer.selectNext("book")) != null) {
        // do something more useful! The element will have all its children elements
        Elements chapters = el.select("chapter");
        // remove this chunk once used to keep DOM light and not run out of memory
        el.remove();
        seenChunks++;
    }

    Document doc = streamer.document(); // the completed doc, will just be a shell
    log("Title", doc.expectFirst("title"));
    log("Seen chunks", seenChunks);
}
}

Parse just the metadata of a website

Assume we are building a link preview tool. All the data we need is in the head section of a page, and so there’s no need to fetch and parse the complete page. Make the request using Jsoup.connect(url), and stream parse it via Response.streamParser().

This example will fetch a given URL, parse only the <head> contents and use those, and then cleanly close the request:

static void selectMeta(String url) throws IOException {
try (StreamParser streamer = Jsoup.connect(url).execute().streamParser()) {
    Element head = streamer.selectFirst("head");
    if (head == null) return;

    log("Title", head.select("title").text());
    log("Description", head.select("meta[name=description]").attr("content"));
    log("Image", head.select("meta[name=twitter:image]").attr("content"));
}
}

Minify the loaded DOM by removing empty text nodes

This example shows a way to progressively parse an input and remove redundant empty textnodes during the parse, resulting in a (somewhat) minified DOM:

static void minifyDocument() {
String html = "<table><tr> <td>a</td> <td>a</td> <td>a</td> <td>a</td> </tr>";
StreamParser streamer = new StreamParser(Parser.htmlParser()).parse(html, "");

streamer.stream()
    .filter(Element::isBlock)
    .forEach(el -> {
        List<TextNode> textNodes = el.textNodes();
        for (TextNode textNode : textNodes) {
            if (textNode.isBlank())
                textNode.remove();
        }
    });

Document minified = streamer.document();
System.out.println(minified.body());
}

Conclusion

The StreamParser provides a practical solution for handling large or streamed XML and HTML documents efficiently, balancing the benefits of both SAX and DOM parsing. Whether you need to extract elements incrementally, reduce memory consumption, or selectively parse content, StreamParser offers a flexible alternative to traditional Java SAX parsers while maintaining the familiar API and robust parsing capabilities of jsoup.

jsoup 1.18.3 quick update

2024-12-02T00:00:00Z

1.18.3 is a quick release to fix #2235 in 1.18.2.

Please see also the full release notes for jsoup 1.18.2 if you are coming from an earlier release.

Download jsoup now.

Bug Fixes

When serializing to XML, attribute names containing -, ., or digits were incorrectly marked as invalid and removed. 2235

If you have any suggestions for the next release, I would love to hear them; please get in touch via jsoup discussions, or with me directly.

You can also follow me (@jhy@tilde.zone) on Mastodon / Fediverse to receive occasional notes about jsoup releases.