Hi! Idiomorph maintainer here. Congrats on the release! It's very, very cool to see how Morphlex is tackling some of the trickier subtleties of DOM morphing. We're in the R&D phase for the next version of Idiomorph, so its awesome to see how you're pushing things forward. I can see some overlap, like with `isEqualNode` (cheers to D* for cluing me into this API). But some of your ideas seem totally fresh, like your approach to solving the reordering problem. I'm looking forward to investigating whether any of these ideas would make sense in Idiomorph, and over all, I'm very pleased to see energy being put into making turn-key SSR more viable!
joeldrapper 18 days ago [-]
Thank you so much. Please ping me if you have any questions about these techniques. I’m `joeldrapper` on Discord and GitHub.
sillysaurusx 22 days ago [-]
I’m curious, what got you interested in solving this particular problem? I.e. what was your specific use case?
Most websites work fine with plain html. If you need something fancier, the world seems to have settled on using React.
I get that this is to let you render html on the backend and then stream it to the site so that JS can update the dom. But why? Genuine question; I’m not saying there’s no good reason.
aatd86 22 days ago [-]
> the world seems to have settled on using React.
The world might have, but I personally have not!!! x(
(I don't think the world really has, the same way the world moved on from jQuery at some point :) and jQuery was probably more widespread)
But if you compare google trends you will find that the crossover point of react vs jQuery was somewhere around 2018.
In other terms, jQuery usage was much more widespread but it is not used for new projects anymore.
harrisi 20 days ago [-]
The trends of Google search doesn't imply anything by itself, just that less people search Google for jQuery than React. Which isn't entirely surprising in my view - people use search engines to learn about something they're unfamiliar with. That doesn't necessarily correlate with increased usage. I've searched for React (although not on Google) but never used it.
It wouldn't be too much work to understand a bit more about the comparative usage. Looking at recent commits of projects on GitHub would be a good start, but also skewed towards open source projects which doesn't represent all actual usage of course.
Another way would be to look through historic changes to websites to see if there's any changes to the source. It'd be a bit complicated because content changes don't necessarily mean anyone is touching jQuery or React pieces.
This also ignores any sort of private usage, which you won't get any reliable data on, and may represent a significant amount of actual usage.
At the end of the day, there's only so much accurate data available to make accurate conclusions about usage of software libraries that don't phone home. The best data available, as far as I'm concerned, is what I posted earlier - and it's still not perfect and doesn't support any claims other than what the data shows.
As a side note, I don't have any dog in this race. I do think it's interesting to get a better understanding of what pieces of software are being used, by whom, in what amount, etc. but it's difficult.
aatd86 20 days ago [-]
No, you see the trends. You see that people have been looking less and less for jQuery and more and more for React.
But React hasn't reached the height of jQuery at its peak.
It is the more reliable proxy.
harrisi 20 days ago [-]
I would not be surprised if you're right in your assertion of what's used more for new projects. I still don't think the evidence you provided is enough to be so certain.
If I want to look up documentation for jQuery, I don't google the term "jquery" to find their docs. I just go straight to the docs directly. For a lot of situations, people's IDEs do enough work to not google something.
aatd86 20 days ago [-]
But that wouldn't still explain why searches for the term has decreased.
Besides, people in general do look up to the online documentation or links to the documentation from stack overflow.
Although the OP created it for SSR, these libraries are handy for SPAs as well.
Rendering the whole DOM tree (instead of VDOMs) is a fast process. The slow part is attaching (committing) elements to the doc. e.g., I have a test of 20,000 elements which takes <30ms to render, while attaching them takes 120ms.
Since the performance is mainly bound to the commit phase, libraries like these (and hopefuly a native API) help for creating simple UI frameworks. For example, a helper such as:
function createElement(tag, props, ...children) {
const elem = document.createElement(tag)
for (const [k, v] of Object.entries(props || {}))
if (k === 'ref') v.elem = elem
else if (k === 'style') Object.assign(elem.style, v)
else if (k.startsWith('on')) elem.addEventListener(k.slice(2).toLowerCase(), ...[v].flat())
else if (k in elem) elem[k] = v
else elem.setAttribute(k, v)
elem.append(...children.flat().filter(Boolean))
return elem
}
could be used, like:
function ResetButton() {
return (
r('button', {
className: CSS.ResetButton,
onClick: store.reset
}, 'Reset'))
}
function render() {
document.body.replaceChildren(App())) // but mergeChildren
}
Both Elixir Phoenix and Ruby on Rails use plain HTML by default but they both support view morphing (phoenix via LiveView and rails via Hotwire Turbo). It really doesn't cost anything to add it. Clicking links with a bit of caching can make it feel near instant the way a (small) SPA does. Adding link prefetching algo on top of the that and it will seem even faster.
If anything it removes a ton of the argument for using React absent maybe a small subset of highly complex UI subcomponents which may need it, but rarely required for a whole SaaS app. Frontend teams just want React so they can use a single tool not because it's the best solution.
joeldrapper 18 days ago [-]
I enjoy writing mostly SSR apps with just a few specific Svelte components mounted as custom elements. It works really well.
nzoschke 22 days ago [-]
I’m doing agentic coding on a bunch of web apps and server side rendering HTML is so much easier than building APIs and React components.
Full page reloads are fine for most CRUD cases. Then layering DOM morphing can be even better UX almost for free
65 21 days ago [-]
I've written SSR SPA frameworks with basic DOM "morphing" - e.g. I need to keep a sidebar from changing the HTML content/state when you click on a link, and I've always found advanced DOM morphing to be sketchy/bug prone and unnecessary.
The way I do it is to update everything _except_ for the DOM nodes that need to be excluded (via data attributes), e.g. the sidebar or a video player. I have found no problems with this approach as I maintain state since the JS is already running before clicking a link, and everything else is updated.
I think this is for if you absolutely have to SSR your markup at all times (e.g. using HTMX), but with something like Alpine.js and using <template> elements, there is no reason to DOM morph.
And like you say, if you need to use crazy advanced DOM morphing, you should probably be using a client side framework anyway. If not, I've gotten away with some very tricky state updates with just Alpine.js.
fpsvogel 21 days ago [-]
Do you have an example of this technique that you can link to, or a fuller discussion of it?
I have a soft spot for Alpine and I’m always on the lookout for things I can do with just Alpine.
joeldrapper 18 days ago [-]
My specific use case was building a form where each change to an input would fetch a new copy of the form from the server and morph it in place.
It means the server-side code can be really simple. You can make parts of the form depend on the values of other parts. For example you can show/hide a section based on a checkbox or fill a select with options based on a previous selection.
Because it was a form, it was really important to maintain object identity and state perfectly so the user would not be interrupted.
hatefulheart 22 days ago [-]
If you read the first 5 sentences of the article you’d see there are at least 3 popular front end libraries that do morphing. I think suggesting the world has settled on anything when it comes to technology is very silly.
*Edit fixed typo.
igor47 21 days ago [-]
Super interesting, I'm just getting into DOM morphing libraries to try to patch around what I've found so far as the main limitation of the htmx approach: losing DOM on swaps. I started a discussion around some of the issues here: https://github.com/bigskysoftware/htmx/discussions/3501
Someone proposed using ideomorph, but it doesn't seem to address the issues I've encountered. Curious if you think a different morphing approach would help?
I've been working on some prototypes for possible immutable DOM APIs and though I haven't gotten that far in experimenting with it I've been expecting to encounter the same problem that this library is designed to solve: in my system the DOM will be represented as a deeply immutable tree of JS objects, arrays, and such, so a state update might consist of being given references to two immutable trees, the current state and the desired state, and from there you need to compute a minimal set of changes so that you don't redo layout and drawing work that doesn't need to be redone for parts of the DOM that are unchanged. This sounds like exactly the algorithm you'd want to do that! So basically it could allow me to use the immutable DOM representation as the source of truth and then diff and sync the new state onto the mutable DOM
ricardobeat 21 days ago [-]
The novelty here is identifying nodes in a list more consistently vs existing options like morphdom. There is a ton of prior art you can draw from, this is called a virtual DOM and is the approach used by React and many contemporary libraries.
conartist6 21 days ago [-]
The vdom idea isn't new to me but doing it with structural identity hashes is a bit different that what React does, no?
jdthedisciple 21 days ago [-]
Wouldn't 'DOM Merging' be a more descriptive term?
kevincox 20 days ago [-]
I don't think merging is an accurate term here. Merging makes me think you take the union of both. But in this case you have two DOM structures and want to make one exactly like the other. And morphing is a good term because you want to change it into the other, not just replace it.
JSR_FDED 21 days ago [-]
The difference is that with morphing they take the trouble of maintaining focus, event listeners, etc.
geon 21 days ago [-]
Is there a library that can work with JSX? I'd like to render JSX on the server and only send the diff to the client. I started writing some experimental code, but it was a lot of work.
Wait but, don't you render the jsx to html? So you you can still perform the diff on the rendered html?
Also, how do you know what you rendered last time? When you do the diff between what's in the browser and what the server just gave you, you have both sides. If you do it server side, you could... Render twice, once before and once after the state change? Or keep a server side cache?
geon 21 days ago [-]
The server would keep a copy of the virtual dom during the session. Each change would be diffed and the diff sent to the client to apply. The client would only apply apply changes from the server, not change the dom on its own, so they wouldn’t get out of sync.
I can render to html, but then I’d need to parse the html back do diff it. Seems stupid.
I don't understand the hate that SPAs get when these are the hoops people will ultimately jump through to render stuff on the server. Maybe you really do have an application at hand, not a "web page" (whatever that is in current year), and then you might as well use the SPA approach to implement it.
joeldrapper 18 days ago [-]
I don’t hate SPAs, I just think some apps are better off being MPAs. I wouldn’t build a todo list app as an MPA. But many apps really are just CRUD forms and tables.
What I don't understand is why you don't just save your own id in memory on all those nodes? It's still not a virtual dom because there is no separate dom tree, you are still just walking the dom itself.
rtcode_io 22 days ago [-]
Can you see if you can support the input-to-output sync of the examples you see on https://rtcode.io ?
Does your library support the new state-preserving moveBefore method?
rtcode_io 22 days ago [-]
Sorry, I was excited to see something newer than diffHTML and asked questions before reading the full article! You do use moveBefore with lots of effort to match elements, which makes Morphlex a very interesting library to try!
I will test your library extensively and update you via GitHub in case of questions/issues!
Just updated to demonstrate what `moveBefore()` brings to the table!!
Try moving the clock.gif line up/down with Alt+↑ and Alt+↓!
(it is very nascent, and I am on a plane with shaky Wi-Fi, so I will give it the love it deserves in a couple of days: compare all libraries with each other, add a benchmark, etc.)
spankalee 21 days ago [-]
I really have questions about this, for two reasons:
1. Coming from a client-side rendering perspective, DOM morphing/diffing is 99% of the time a bad idea, except in the case of reordering a list of keyed items where you can use a simpler, more specialized algorithm.
It's much better to use template identity to compare the source template of the current DOM with the source template of the incoming DOM (or description of DOM) and completely re-render if the source template changed. It's a very simple and fast check, and nearly all the time you change templates you want new DOM state anyway.
This technique works with SSR'ed HTML as well. You leave marker comments in the HTML that bracket the nodes created from a template and carry with them a template ID (e.g. a hash of the template). When updating the DOM, as you traverse the template instance tree, you check IDs and replace or update in-place as needed. Again, simple and fast.
2. But... If you're morphing the existing DOM, this seems to eliminate many of the benefits of sending HTML in your server responses in the first place. The HTML is just data at that point - you parse it only to crawl it as a set of instructions for updating the DOM.
HTML is an expensive way to do this. It's a larger format and slower to parse than JSON, and then you have to do this diffing. You'd be better off doing client-side rendering if possible. Data + templates is usually a very compressed format compared to the already expanded HTML you get from rendering the templates client-side.
And if the reason to morph is to keep things like event listeners, templates would let you attach those to the new DOM as well as preserve them in the unchanged DOM. With DOM morphing you need a way to go set things up on the new DOM anyway.
...
The big advantage of this is the architectural simplicity of only ever returning HTML from the server, as opposed to HTML for first render and data for updates, but it's not going to have good network and rendering perf compared to CSR for updates.
ricardobeat 21 days ago [-]
The reason simple identity is not "better", and the whole reason these libraries and React's virtual DOM exist, is that the DOM is stateful.
This approach works for simple stuff, until it doesn't. Form inputs will lose values, focus will be lost (ruining accessibility in the process), videos will restart, etc. You need the diffing to prevent unnecessary changes to the DOM. Even worse, in complex applications you easily end up in situations where the trivial approach causes vast swathes of the page to rerender at once, either because of unplanned dependencies or simply because you have three, seven or forty teams working on the site at the same time.
spankalee 21 days ago [-]
I think you misunderstand what I'm saying. If the template identity is the same, you _don't_ replace the DOM - you just update the bound values in the template if necessary. If those are nested templates, you recurse and apply the same logic. This keeps the DOM stable when updating repeatedly from a template, even in very complex applications.
From experience, this works in apps like photo editors, video platforms, forums, app stores, home automation, application builders... It's just extremely rare to have two totally different templates with a shared element in them that you want to keep stable - the literally 99.9% case is that if the template identity changes, the DOM should be cleared.
ricardobeat 21 days ago [-]
Ah, I did misread your comment. What you describe is conceptually similar to Svelte's approach in the client, or even signals; keeping a reference directly to the node that used a certain value, though the client-side libraries have the luxury of keeping a pointer to the actual node.
With DOM state being thrown away, it would still not be possible to build as-you-type input validation for example. For SSR + streaming server updates I get the feeling it would also have limited utility, how do you track dependencies across more complex template conditions, loops? Is querying for comment markers any faster than traversing DOM elements? If using generated IDs, do you keep node IDs in memory for each user session in the server when dealing with dynamic content? Are you using an existing open source solution for this?
The DOM diffing/morphing approach is popular because it's in fact extremely fast to run, has small memory requirements, and is a low complexity implementation. In the SSR case, you don't need anything special on the server side, it can be completely ignorant of what is happening in the client. It's hard to beat.
joeldrapper 18 days ago [-]
It can be a mistake though to assume that the DOM hasn’t changed since it was rendered. Browser extensions, ad blockers and other JavaScript can modify the DOM.
I know it’s more expensive, but it’s like 1ms to render a document on the server and 3ms to morph it in the client. If you keep an SSE connection open, Brotli compression is very effective when you send almost the same HTML again and again.
andersmurphy 21 days ago [-]
This garbage demo uses a morphing library (Datastar which may switch to using morplex in future) to morph in around 12k divs per frame on any change by any user.
The event listener part is easy use a top level event handler and bubble up events.
The network part is also easy brotli compression over an SSE stream. Even though this demo returns around 180kb per frame uncompressed, a single check between frames will compress to 13bytes on the wire.
We already asked you barely more than a month ago to avoid posting flamewar style comments on HN. This is only a place where people want to participate because others make the effort to raise the standards, not drag them down. Please try to be one of the ones to make this place better not worse, otherwise find somewhere else that's more welcoming of that style of behaviour. https://news.ycombinator.com/newsguidelines.html
JSR_FDED 21 days ago [-]
How about both of you show metrics so this becomes a fact based discussion?
sudodevnull 21 days ago [-]
Look at any Datastar demo, updates in microseconds above half RTT. Look at Andrew's demo above. We are actually working with Joel on possibly moving our already fastest approach to morphing to a version of his morphlex work. Actually try it and measure for yourself
spankalee 21 days ago [-]
"Sit back down"? What kind of child are you?
sudodevnull 21 days ago [-]
So much misunderstanding of the details without actually trying it. I said clearly if you have metrics to back up claims great! Otherwise it's pure FUD which goes against actual metrics in the wild. Back up your assertions with actual code, that'd be great since you are so confident its 99% wrong.
tomhow 21 days ago [-]
Please don't reply to a bad comment with another bad one, it just drags things down further.
Rendered at 11:08:43 GMT+0000 (Coordinated Universal Time) with Vercel.
Most websites work fine with plain html. If you need something fancier, the world seems to have settled on using React.
I get that this is to let you render html on the backend and then stream it to the site so that JS can update the dom. But why? Genuine question; I’m not saying there’s no good reason.
The world might have, but I personally have not!!! x(
(I don't think the world really has, the same way the world moved on from jQuery at some point :) and jQuery was probably more widespread)
It wouldn't be too much work to understand a bit more about the comparative usage. Looking at recent commits of projects on GitHub would be a good start, but also skewed towards open source projects which doesn't represent all actual usage of course.
Another way would be to look through historic changes to websites to see if there's any changes to the source. It'd be a bit complicated because content changes don't necessarily mean anyone is touching jQuery or React pieces.
This also ignores any sort of private usage, which you won't get any reliable data on, and may represent a significant amount of actual usage.
At the end of the day, there's only so much accurate data available to make accurate conclusions about usage of software libraries that don't phone home. The best data available, as far as I'm concerned, is what I posted earlier - and it's still not perfect and doesn't support any claims other than what the data shows.
As a side note, I don't have any dog in this race. I do think it's interesting to get a better understanding of what pieces of software are being used, by whom, in what amount, etc. but it's difficult.
It is the more reliable proxy.
If I want to look up documentation for jQuery, I don't google the term "jquery" to find their docs. I just go straight to the docs directly. For a lot of situations, people's IDEs do enough work to not google something.
Seems pretty obvious looking at the graph: https://trends.google.com/trends/explore?date=all&q=%2Fm%2F0...
Rendering the whole DOM tree (instead of VDOMs) is a fast process. The slow part is attaching (committing) elements to the doc. e.g., I have a test of 20,000 elements which takes <30ms to render, while attaching them takes 120ms.
Since the performance is mainly bound to the commit phase, libraries like these (and hopefuly a native API) help for creating simple UI frameworks. For example, a helper such as:
could be used, like: Here's an example of using that helper:https://github.com/ericfortis/mockaton/blob/main/src/client/...
If anything it removes a ton of the argument for using React absent maybe a small subset of highly complex UI subcomponents which may need it, but rarely required for a whole SaaS app. Frontend teams just want React so they can use a single tool not because it's the best solution.
Full page reloads are fine for most CRUD cases. Then layering DOM morphing can be even better UX almost for free
The way I do it is to update everything _except_ for the DOM nodes that need to be excluded (via data attributes), e.g. the sidebar or a video player. I have found no problems with this approach as I maintain state since the JS is already running before clicking a link, and everything else is updated.
I think this is for if you absolutely have to SSR your markup at all times (e.g. using HTMX), but with something like Alpine.js and using <template> elements, there is no reason to DOM morph.
And like you say, if you need to use crazy advanced DOM morphing, you should probably be using a client side framework anyway. If not, I've gotten away with some very tricky state updates with just Alpine.js.
I have a soft spot for Alpine and I’m always on the lookout for things I can do with just Alpine.
It means the server-side code can be really simple. You can make parts of the form depend on the values of other parts. For example you can show/hide a section based on a checkbox or fill a select with options based on a previous selection.
Because it was a form, it was really important to maintain object identity and state perfectly so the user would not be interrupted.
*Edit fixed typo.
Someone proposed using ideomorph, but it doesn't seem to address the issues I've encountered. Curious if you think a different morphing approach would help?
(DOM Morph RealTime HyperText)
I've been working on some prototypes for possible immutable DOM APIs and though I haven't gotten that far in experimenting with it I've been expecting to encounter the same problem that this library is designed to solve: in my system the DOM will be represented as a deeply immutable tree of JS objects, arrays, and such, so a state update might consist of being given references to two immutable trees, the current state and the desired state, and from there you need to compute a minimal set of changes so that you don't redo layout and drawing work that doesn't need to be redone for parts of the DOM that are unchanged. This sounds like exactly the algorithm you'd want to do that! So basically it could allow me to use the immutable DOM representation as the source of truth and then diff and sync the new state onto the mutable DOM
https://github.com/geon/react-node-diff/blob/main/src/diff-r...
Also, how do you know what you rendered last time? When you do the diff between what's in the browser and what the server just gave you, you have both sides. If you do it server side, you could... Render twice, once before and once after the state change? Or keep a server side cache?
I can render to html, but then I’d need to parse the html back do diff it. Seems stupid.
I tried building a redux-style spa like that. Worked fine. https://github.com/geon/server-side-spa/tree/main/src/server
I looked it up on Github and they seem to be using the idiomorph package.
[1] https://dev.37signals.com/turbo-8-released/
Does your library support the new state-preserving moveBefore method?
I will test your library extensively and update you via GitHub in case of questions/issues!
Thank you for releasing Morphlex!
To see `moveBefore()` in action:
try moving the clock.gif line up/down with Alt+↑ and Alt+↓!
https://dm.rt.ht/perf
Morphlex is 8→69× faster and 4× smaller!
The playground you asked for has already helped me file and fix a bug https://github.com/yippee-fun/morphlex/issues/38 in my brief review of Morphlex.
See https://dm.rt.ht
Just updated to demonstrate what `moveBefore()` brings to the table!!
Try moving the clock.gif line up/down with Alt+↑ and Alt+↓!
(it is very nascent, and I am on a plane with shaky Wi-Fi, so I will give it the love it deserves in a couple of days: compare all libraries with each other, add a benchmark, etc.)
1. Coming from a client-side rendering perspective, DOM morphing/diffing is 99% of the time a bad idea, except in the case of reordering a list of keyed items where you can use a simpler, more specialized algorithm.
It's much better to use template identity to compare the source template of the current DOM with the source template of the incoming DOM (or description of DOM) and completely re-render if the source template changed. It's a very simple and fast check, and nearly all the time you change templates you want new DOM state anyway.
This technique works with SSR'ed HTML as well. You leave marker comments in the HTML that bracket the nodes created from a template and carry with them a template ID (e.g. a hash of the template). When updating the DOM, as you traverse the template instance tree, you check IDs and replace or update in-place as needed. Again, simple and fast.
2. But... If you're morphing the existing DOM, this seems to eliminate many of the benefits of sending HTML in your server responses in the first place. The HTML is just data at that point - you parse it only to crawl it as a set of instructions for updating the DOM.
HTML is an expensive way to do this. It's a larger format and slower to parse than JSON, and then you have to do this diffing. You'd be better off doing client-side rendering if possible. Data + templates is usually a very compressed format compared to the already expanded HTML you get from rendering the templates client-side.
And if the reason to morph is to keep things like event listeners, templates would let you attach those to the new DOM as well as preserve them in the unchanged DOM. With DOM morphing you need a way to go set things up on the new DOM anyway.
...
The big advantage of this is the architectural simplicity of only ever returning HTML from the server, as opposed to HTML for first render and data for updates, but it's not going to have good network and rendering perf compared to CSR for updates.
This approach works for simple stuff, until it doesn't. Form inputs will lose values, focus will be lost (ruining accessibility in the process), videos will restart, etc. You need the diffing to prevent unnecessary changes to the DOM. Even worse, in complex applications you easily end up in situations where the trivial approach causes vast swathes of the page to rerender at once, either because of unplanned dependencies or simply because you have three, seven or forty teams working on the site at the same time.
From experience, this works in apps like photo editors, video platforms, forums, app stores, home automation, application builders... It's just extremely rare to have two totally different templates with a shared element in them that you want to keep stable - the literally 99.9% case is that if the template identity changes, the DOM should be cleared.
With DOM state being thrown away, it would still not be possible to build as-you-type input validation for example. For SSR + streaming server updates I get the feeling it would also have limited utility, how do you track dependencies across more complex template conditions, loops? Is querying for comment markers any faster than traversing DOM elements? If using generated IDs, do you keep node IDs in memory for each user session in the server when dealing with dynamic content? Are you using an existing open source solution for this?
The DOM diffing/morphing approach is popular because it's in fact extremely fast to run, has small memory requirements, and is a low complexity implementation. In the SSR case, you don't need anything special on the server side, it can be completely ignorant of what is happening in the client. It's hard to beat.
I know it’s more expensive, but it’s like 1ms to render a document on the server and 3ms to morph it in the client. If you keep an SSE connection open, Brotli compression is very effective when you send almost the same HTML again and again.
The event listener part is easy use a top level event handler and bubble up events.
The network part is also easy brotli compression over an SSE stream. Even though this demo returns around 180kb per frame uncompressed, a single check between frames will compress to 13bytes on the wire.
https://checkboxes.andersmurphy.com/
See https://news.ycombinator.com/item?id=45941553