Design questions around pay-for-what-you-use

[kenchris]

There seems to be a desire to provide more use-specific elements to the platform. Pay for what you use, it a way to enable that by only letting the users of these specific elements pay for the additional loading time and memory usage.

There are comments suggesting that, if implemented in native browser code, the host OS might be smart enough to mitigate this, ie. it will just load the memory regions when needed.

That might not be the case if some of these elements are written as ES modules internally in the browser, or used as polyfills initially in order to get proper web developer feedback.

Related to:

whatwg/html#4696
whatwg/html#4697
#384
#385

[kenchris]

This ties into the desire to make the web platform more approachable from a developer point of view, in multiple ways.

For instance, when a new developer is tasked to create an Android or iOS application for the first time, it is quite approachable in the sense that you can just follow the tutorials and use the builtin elements which covers most common use-cases. These platforms have a wider vocabulary of elements and utilities that gets you pretty far. That is not the case on the web where you need to create most from scratch or use frameworks.

Talking to enterprise companies, I have been hearing that that is a big issue when designing software that needs to last for years. Relying on multiple libraries brings challenges:

• What is the license?
• How is it maintained? Will we have to maintain it in a couple of years or migrate to something else?
• What other dependencies does it have? Is it safe? Will it pull in other deps in the future behind my back?

Angular seems to be particular popular in the enterprise because it comes with 'all batteries included' and everything is under the same license and maintenance model, thus easier for companies to deal with.

Broadening the web platform with more specific elements without making non-users pay for it seems like a valuable goal, but it is also super important that these new elements have gotten battle tested and used by actual users (feedback loop).

I assume that that is part of the idea with built-in modules and import maps, because users can start using a polyfilled version (maybe even versioned) that is being developed together with the spec and when/if these prove good and popular (look at usage?) then can be upgraded to become part of HTML or similar standards.

[hober]

It's unclear to me if this is a direction anyone is still pursuing, given tc39/Reflector#247 and whatwg/webidl#675. Has this design review been overtaken by events?

[kenchris]

@domenic @littledan @annevk might be able to shed some light

[domenic]

I am no longer working on built-in modules, and in fact believe them to be harmful and a subpar way of achieving pay-for-what-you-use. I am attempting to see if anyone at Google remains interested in some way so they can report back.

[littledan]

I believe built-in modules should be a valuable addition to the web platform, and I'm really excited at @msaboff and @mattijs's efforts for this in TC39 (cc @syg @erights). I've closed the WebIDL PR, as it's a bit early with respect to that more foundational effort. It seems to me that we'll have to learn more about how the TC39 proposal evolves before we can consider changing conventions to support built-in modules, as was previously proposed.

[domenic]

Let me post for the TAG's consideration my stance on built-in modules, adapted the private reflector thread above:

I believe that built-in modules are harmful and should not be pursued. This realization came after two years of leading a team within Chrome that explored using them for the web, so it is hard-fought. What follows is my personal opinion; as I am no longer working in this area, the Chrome team's position may be different. (But to the extent I can influence it in this direction, I will.)

One of the first things you notice about built-in modules is that they are confusing. JavaScript developers often believe that by virtue of new APIs being in a module, they are part of a new "standard library" effort, somehow distinct from all of the previous standard library work that landed as built-in globals. It's never clear exactly what they believe to be different; in particular, few seem to realize that the difference is only in the exposure technology and syntax. We won't move any faster in building the standard library if we switch from putting it in globals to putting it in modules. We won't stop needing specifications, and consensus, and tests. The only difference is that new classes designed after some point in time will end up accessed one way, and classes designed before another.

However, built-in modules become technically problematic when you realize that we have to re-solve all of the things we've already figured out with built-in globals. This includes things such as: loading things needed for application startup synchronously; how to polyfill; how to virtualize; how to name things; and how to integrate with the rest of the ecosystem (e.g. a new temporal module integrating with the Intl global). My team pursued answers to these questions for a long time. But eventually we realized that we were piling complexity on top of complexity, only to re-achieve the same capabilities we already get with built-in globals. When viewed from that angle, we decided to drop all that complexity (e.g. removing support for built-in module fallback from import maps).

Built-in modules are also dangerous in that they are being used as an opportunity to relitigate decisions that support the foundation of the web today. These include attempts to move away from the shared cooperative namespace of the global object; to introduce a new governance structure for names; to cripple the flexibility and polyfillability of the platform via freezing or cons-ing up new objects; or the introduction of developer-facing versioning to a versionless language and standard library. Although I too was excited about the opportunity to break slightly with the past and refine API definitions, what it took some time to realize is that others were seeing this as an opportunity to make much more drastic and harmful changes.

With all of these drawbacks, built-in modules would need a lot of compelling advantages to still be worthwhile. But what our team has found is that those advantages are chimeral. The problem of a "polluted global namespace" is largely a non-problem; the only global introduced by specifications that has had conflict with developer code is globalThis. A second global namespace of module specifiers, while theoretically cleaner in that it doesn't collide with global variable declarations, in practice will have the same potential collisions---unless you also take some of the more dangerous suggestions alluded to above and cripple polyfillability. Shorter post-import names is another surface advantage, but this can be accomplished just as easily with namespace objects. My team explored writing built-in modules in JavaScript instead of C++, but we could have (and Chrome has in the past) written built-in globals this way; modules do not enable anything new here.

The only potentially-compelling advantage of built-in modules is that, because importing is necessarily async, they might be able to be lazy-loaded, or "pay for what you use". But this, too, is not very accurate. When I mentioned this idea to Eric Rescola (Firefox CTO) when my team was first getting started, he pointed out that lazy-loading is also possible for globals, just by hiding the APIs behind promises (cf. specs like navigator.getBattery()). And when we brought up this idea on whatwg/html, Ryosuke Niwa from Apple's WebKit team pointed out that modern OS kernels are smart enough that it won't spend memory on unused APIs anyway, and in fact dynamically loading built-in modules could create drawbacks due to each realm bringing in different modules, causing at least WebKit/JSC to end up with more "dirty memory".

Overall, the benefits of built-in modules are slight or nonexistent, and their drawbacks are substantial. As such, I urge both TC39 and the wider web standards community to stand strong against the idea of bifurcating the web standard library into globals-before-2020, modules-after-2020, and instead stick with globals. If there is a desire to change aspects of how the standard library is specified, e.g. hiding more APIs behind promises, using namespace more extensively, or collaborating between TC39 and the Web IDL editors to align on conventions like enumerability, those can all be pursued while continuing to use globals. Built-in modules do not bring enough to the table to outweigh their dangers, and the module system should remain the purview of user-space libraries or specific hosts like Node.js.

I hope the TAG finds this perspective valuable.

[littledan]

Note, Ecma and the W3C have a bilateral liaison relationship, so if any TAG members want to access the Reflector, attend TC39 meetings in liaison capacity, etc, I believe this should be possible; just let me know and I'll help you through the process.

[dbaron]

@alice and I are looking at this in a breakout at the TAG's Wellington face-to-face meeting.

Domenic's conclusion in #421 (comment) seems like a reasonable one to me, although I'd note that abandoning built-in modules doesn't mean abandoning work towards the idea that more parts of the web platform should be pay-for-what-you-use. There still seems to be open discussion in whatwg/html#4696 and elsewhere.

That said, I'm not sure what the goal of keeping this issue open is. The design questions for which we opened it seem like they are likely no longer relevant.

Feel free to open another issue if you'd like further TAG feedback or discussion on issues related to this topic.