HTML API: Rely on HTML API for `wp_html_split()` #6651

dmsnell · 2024-05-27T17:30:41Z

Status

This is a work in progress.

Special atomic elements need deeper access to split apart because the token and text boundaries aren't exposed. Can we reconstruct this by changing the start tag name and re-parsing?

Description

Replace the regular-expression approach to splitting HTML with the HTML API for a reliable parse.

(cherry picked from commit b7b2876)

(cherry picked from commit b3b4562)

(cherry picked from commit 57e737c)

(cherry picked from commit cdf7462)

github-actions · 2024-05-27T17:30:58Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props dmsnell, jonsurrell.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

github-actions · 2024-05-27T17:30:59Z

Trac Ticket Missing

This pull request is missing a link to a Trac ticket. For a contribution to be considered, there must be a corresponding ticket in Trac.

To attach a pull request to a Trac ticket, please include the ticket's full URL in your pull request description. More information about contributing to WordPress on GitHub can be found in the Core Handbook.

github-actions · 2024-05-27T17:43:59Z

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

The Plugin and Theme Directories cannot be accessed within Playground.
All changes will be lost when closing a tab with a Playground instance.
All changes will be lost when refreshing the page.
A fresh instance is created each time the link below is clicked.
Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

sirreal · 2024-05-28T13:49:26Z

It seems like all the remaining test failures are around CDATA sections. The assertions appear to be wrong in HTML5. @westonruter shared some stats on XHTML usage and there seems to be very little: GoogleChromeLabs/wpp-research#74

Should theses tests be removed or updated since they're wrong in the majority of cases?

It would be great to see https://core.trac.wordpress.org/ticket/59883 (drop support for pre-HTML5) move forward.

sirreal · 2024-05-28T14:57:51Z

There is one test failure I see that's not CDATA related, but it also seems like it may be a fix in behavior:

1) WP_Test_REST_Comments_Controller::test_comment_roundtrip_as_superadmin
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'<p>\\&#038;\\ &amp; &invalid; < &lt; &amp;lt;
-</p>'
+'<p>\\&#038;\\ &amp; &invalid; < &lt; &amp;lt;</p>'

westonruter

Should wp_get_internal_tag_processor() be marked as private? Or else, should the logic be put in a closure?

westonruter · 2024-05-28T17:07:36Z

src/wp-includes/formatting.php

 */
-function wp_html_split( $input ) {
-	return preg_split( get_html_split_regex(), $input, -1, PREG_SPLIT_DELIM_CAPTURE );
+function wp_html_split( $input_html ) {


Suggested change

function wp_html_split( $input_html ) {

$get_internal_tag_processor = static function ( $html ) {

return new class( $html ) extends WP_HTML_Tag_Processor {

/**

* Returns the raw token from the input string at the

* current location, if paused at a location.

*

* @return false|string

*/

public function get_raw_token() {

if (

WP_HTML_Tag_Processor::STATE_READY === $this->parser_state ||

WP_HTML_Tag_Processor::STATE_INCOMPLETE_INPUT === $this->parser_state ||

WP_HTML_Tag_Processor::STATE_COMPLETE === $this->parser_state

) {

return false;

}

$this->set_bookmark( 'here' );

$here = $this->bookmarks['here'];

return substr( $this->html, $here->start, $here->length );

}

};

};

westonruter · 2024-05-28T17:07:54Z

src/wp-includes/formatting.php

-	return preg_split( get_html_split_regex(), $input, -1, PREG_SPLIT_DELIM_CAPTURE );
+function wp_html_split( $input_html ) {
+	$chunks    = array();
+	$processor = wp_get_internal_tag_processor( $input_html );


Suggested change

$processor = wp_get_internal_tag_processor( $input_html );

$processor = $get_internal_tag_processor( $input_html );

westonruter · 2024-05-28T17:08:00Z

src/wp-includes/formatting.php

+		$raw_html    = $processor->get_raw_token();
+		$first_char  = $raw_html[1];
+		$raw_html[1] = 'X';
+		$special     = wp_get_internal_tag_processor( $raw_html );


Suggested change

$special = wp_get_internal_tag_processor( $raw_html );

$special = $get_internal_tag_processor( $raw_html );

westonruter · 2024-05-28T17:08:11Z

src/wp-includes/formatting.php

+/**
+ * Returns a Tag Processor exposing the raw matched tokens.
+ *
+ * @since 6.6.0
+ *
+ * @param string $html Passed into the Tag Processor.
+ * @return WP_HTML_Tag_Processor|__anonymous@23567
+ */
+function wp_get_internal_tag_processor( $html ) {
+	return new class( $html ) extends WP_HTML_Tag_Processor {
+		/**
+		 * Returns the raw token from the input string at the
+		 * current location, if paused at a location.
+		 *
+		 * @return false|string
+		 */
+		public function get_raw_token() {
+			if (
+				WP_HTML_Tag_Processor::STATE_READY === $this->parser_state ||
+				WP_HTML_Tag_Processor::STATE_INCOMPLETE_INPUT === $this->parser_state ||
+				WP_HTML_Tag_Processor::STATE_COMPLETE === $this->parser_state
+			) {
+				return false;
+			}
+
+			$this->set_bookmark( 'here' );
+			$here = $this->bookmarks['here'];
+
+			return substr( $this->html, $here->start, $here->length );
+		}
+	};
+}
+


Suggested change

/**

* Returns a Tag Processor exposing the raw matched tokens.

*

* @since 6.6.0

*

* @param string $html Passed into the Tag Processor.

* @return WP_HTML_Tag_Processor|__anonymous@23567

*/

function wp_get_internal_tag_processor( $html ) {

return new class( $html ) extends WP_HTML_Tag_Processor {

/**

* Returns the raw token from the input string at the

* current location, if paused at a location.

*

* @return false|string

*/

public function get_raw_token() {

if (

WP_HTML_Tag_Processor::STATE_READY === $this->parser_state ||

WP_HTML_Tag_Processor::STATE_INCOMPLETE_INPUT === $this->parser_state ||

WP_HTML_Tag_Processor::STATE_COMPLETE === $this->parser_state

) {

return false;

}

$this->set_bookmark( 'here' );

$here = $this->bookmarks['here'];

return substr( $this->html, $here->start, $here->length );

}

};

}

sirreal · 2025-07-29T11:24:14Z

Is this superseded by #9270?

dmsnell · 2025-07-29T15:04:41Z

Yes @sirreal — thanks. I should have found this and linked it when I created #9270. I’ve updated its PR description accordingly.

Not ready to close this though, as I still will probably want to merge ideas from both of them.

dmsnell and others added 5 commits May 27, 2024 10:20

HTML API: Rewrite wp_html_split relying on the HTML API.

8a0862c

Fix off-by-1

bc1c877

(cherry picked from commit b7b2876)

Fix interactivity

89d28fa

(cherry picked from commit b3b4562)

Remove off-by-1 adjustment

8fa3657

(cherry picked from commit 57e737c)

Fix tag token length error

92f2c48

(cherry picked from commit cdf7462)

dmsnell mentioned this pull request May 27, 2024

Greater-than-sign in attribute values cause other special characters to be encoded as HTML entities inside block's render php WordPress/gutenberg#56597

Closed

dmsnell added 3 commits May 27, 2024 11:07

Properly split special atomic elements.

53b18ee

Rerarrange cod

7670886

Add text nodes where missing, to preserve legacy behavior.

baefd6e

dmsnell mentioned this pull request May 27, 2024

HTML API: Fix tag processor token length #6625

Closed

Remove off-by-1 adjustment

db9c494

westonruter reviewed May 28, 2024

View reviewed changes

dmsnell mentioned this pull request Jun 9, 2024

Dennis' list of broad and interesting things. WordPress/gutenberg#62437

Open

sirreal marked this pull request as draft July 29, 2025 11:23

dmsnell mentioned this pull request Jul 29, 2025

HTML API: Reliably parse HTML in wp_html_split(). #9270

Draft

3 tasks

dmsnell mentioned this pull request Sep 11, 2025

HTML API: Refactor uses of wp_html_split() #9851

Draft

	$processor = wp_get_internal_tag_processor( $input_html );
	$processor = $get_internal_tag_processor( $input_html );

	$special = wp_get_internal_tag_processor( $raw_html );
	$special = $get_internal_tag_processor( $raw_html );

HTML API: Rely on HTML API for wp_html_split() #6651

Are you sure you want to change the base?

HTML API: Rely on HTML API for wp_html_split() #6651

Uh oh!

Conversation

dmsnell commented May 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Status

Description

Uh oh!

github-actions bot commented May 27, 2024

Uh oh!

github-actions bot commented May 27, 2024

Trac Ticket Missing

Uh oh!

github-actions bot commented May 27, 2024

Test using WordPress Playground

Some things to be aware of

Uh oh!

sirreal commented May 28, 2024

Uh oh!

sirreal commented May 28, 2024

Uh oh!

westonruter left a comment

Choose a reason for hiding this comment

Uh oh!

westonruter May 28, 2024

Choose a reason for hiding this comment

Uh oh!

westonruter May 28, 2024

Choose a reason for hiding this comment

Uh oh!

westonruter May 28, 2024

Choose a reason for hiding this comment

Uh oh!

westonruter May 28, 2024

Choose a reason for hiding this comment

Uh oh!

sirreal commented Jul 29, 2025

Uh oh!

dmsnell commented Jul 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HTML API: Rely on HTML API for `wp_html_split()` #6651

HTML API: Rely on HTML API for `wp_html_split()` #6651

dmsnell commented May 27, 2024 •

edited

Loading