jsoup CSS query migration guide

jsoup’s CSS queries are changing to follow the CSS and Web DOM standards more closely. jsoup 1.24.1 introduces Web mode for these changes and retains the behavior of earlier releases as legacy mode during the transition. Most queries will continue to work without changes.

As CSS and the Web DOM have evolved, jsoup has preserved its original query behavior for backwards compatibility. Over time, the differences between jsoup’s behavior and those standards have grown. The transition to Web mode brings them closer together, with a staged migration that gives you time to check your code and update affected queries before the default changes.

In legacy mode, select() and selectFirst() can return the element you call them on if it matches the query. For example, calling select("div") on a div element would include that element in the returned Elements. Web mode limits results to matching descendants, excluding the starting element. Queries will also follow HTML and XML case-matching rules, so comparisons that previously ignored case may now require an exact match.

Web mode also changes [attr~=value] from a regular-expression comparison to CSS word matching. For example, [class~=card] matches class="card featured", but not class="cardinal". Attribute regular expressions remain available through the new :matchesAttr() selector.

Queries such as doc.select("div") still return the same elements. The change to descendant-only selection matters when querying from an element that could itself match, such as products.select("div"). Queries that rely on case-insensitive matching or attribute regexes may also need to be updated.

Test and migrate your code

jsoup 1.24.1 introduces jsoup.queryMode, a system property that lets you choose the query mode during migration.

Version Default Available settings
1.24.1 legacy legacy, web, check
1.25.1 web legacy, web, check
1.26.1 onward Web behavior only Setting removed

Start with check: run your tests with compatibility checks enabled.

java -Djsoup.queryMode=check -jar your-app.jar

Or set the property in code before running your queries:

System.setProperty("jsoup.queryMode", "check");

The check mode lets you preview the change before switching to Web mode. jsoup will execute the query in both modes, return the legacy results, and report any differences between the two to stderr. You can review the differences and update affected queries, then move your application to jsoup.queryMode=web.

In 1.25.1, the default will change to web. jsoup.queryMode=legacy provides a temporary way to retain the original behavior while migrating. In 1.26.1 the setting will no longer take effect as the legacy mode will be removed.

Selecting from an element

Web mode follows the Web DOM’s querySelectorAll() and querySelector() methods: match descendants only. Descendants are the elements nested inside the element you query from. In Web mode, this rule also applies to getElementById(String), getElementsBy*(), and getAllElements().

For example, consider a div containing another div:

Element products = Jsoup.parse(
    "<div id=products><div class=product>Tea</div></div>"
).getElementById("products");

Elements items = products.select("div");

In legacy mode, select("div") returns both divs; in Web mode, it returns only the inner .product div. Similarly, selectFirst("div") returns the container in legacy mode and the inner div in Web mode.

In Web mode, selectors can refer to ancestors outside the container. For example, products.select("body div") returns the inner product div because it has a body ancestor, even though that ancestor is outside products. Only descendants of products are returned.

If you need to include the container itself in the results, see Query results and scope.

In either mode, is() tests the element itself against the query. closest() starts with that element and walks up its ancestors, returning the first match.

Case matching

Legacy mode ignores case in class selectors and most attribute comparisons. Web mode makes these comparisons case-sensitive by default, though you can still specify case-insensitive matching if desired.

The matching rules also depend on how the document was parsed. jsoup uses HTML rules by default, or XML rules for documents parsed with Parser.xmlParser().

In Web mode, attribute names and values follow different rules. For example, in [data-sku="AbC123"], data-sku is the name and AbC123 is the value. HTML tag and attribute names ignore ASCII case, so [DATA-SKU] and [data-sku] are equivalent. In XML, names must match exactly.

Attribute values are generally case-sensitive: [data-sku="AbC123"] won’t match data-sku="abc123". Some HTML-defined values, such as type and rel, remain case-insensitive by default. To choose how values are compared, add a flag:

Selector Value comparison in Web mode
[data-sku="AbC123"] Uses the default rules; exact for data-sku.
[data-sku="AbC123" i] Ignores ASCII case.
[type="TEXT" s] Requires exact case, overriding the HTML default.

The i and s flags affect attribute values only. To match multiple variants of an XML attribute name, include each explicitly: [data-sku], [DATA-SKU].

Both flags are supported in each query mode from 1.24.1, so you can update queries that rely on case-insensitive matching before switching to Web mode. For example, [href*="/Products/"] won’t match href="/products/tea" in Web mode. Add the i flag to preserve that match: [href*="/Products/" i].

For case-insensitive class matching in Web mode, use [class~="product" i] instead of .product.

These case-matching rules also apply to jsoup’s class and attribute finders, and to hasClass(). getElementById() remains case-sensitive in both modes.

Attribute regexes

jsoup has historically used ~= to match attribute values against regular expressions. In Web mode, it now follows the CSS meaning: matching a whole word in a whitespace-separated attribute value. For example, [class~=card] matches class="featured card", but not class="postcard".

For regular expressions, use the new :matchesAttr() selector. It’s available in every query mode from 1.24.1, so you can update your queries before switching to Web mode:

// Legacy mode
document.select("img[src~=(?i)\\.(png|jpe?g)]");

// All modes from 1.24.1
document.select("img:matchesAttr(src, (?i)\\.(png|jpe?g))");

Compatibility testing

With jsoup.queryMode=check, jsoup runs each supported query in both legacy and Web modes, compares the results, and returns only the legacy results. If those results differ, or the query fails in Web mode, jsoup will print a diagnostic to stderr.

jsoup query compat: select("div"): legacy=2, web=1; results differ; at example.ProductParser.items(ProductParser.java:42); at example.CatalogJob.run(CatalogJob.java:18)

Here, select("div") returns two elements in legacy mode but only one in Web mode. The diagnostic includes the source location so you can find the query and review the difference.

Compatibility checks cover common string-query methods, including select() and selectFirst(). They don’t cover streams, node queries, supplied evaluators, or getElement* finders; see the full checking coverage.

Running both modes adds overhead, so use this setting for compatibility testing rather than normal production use.

Further help

The Selector compatibility reference covers the full matching rules, including how selectors can match ancestors outside the element you query from. If you find an unexpected difference, please share a small input document, your query, and your expected result in jsoup discussions.