Introduction ------------- There was [a recent announcement](https://web.dev/import-maps-in-all-modern-browsers/) about a milestone -- all major browsers [now support](https://caniuse.com/import-maps) ECMAScript Modules (ESM) with "import-maps". Import-maps make ESM viable for complex applications (like CiviCRM) without requiring NodeJS during deployment. The aim of this issue is to fully support ESM for JS development in CiviCRM -- by leveraging browser support for import-maps. It is broken down into a few sections: * Part 1: Basic ESM Support * Part 2: Import Map Support However, because this is relatively new functionality with some risk and compatibility issues, the approach described here is an incremental phase-in -- where specific files/pages/extensions may use ESM (without requiring it for existing screens). Background ---------- * As discussed in https://gist.github.com/totten/5c34e3885a4fe7002f990e09395b4294, one major factor in updating CiviCRM's Javascript tooling is support for ECMAScript Modules (ESM). * ESM has increasingly become necessary for a wide of range Javascript tools and libraries. * Originally, browsers did not support ESM. Instead, JS developers used NodeJS-based tools (such as Webpack or Rollup) to convert ESM's into basic Javascript. However, browser support for ESM has improved over the years. * For a decent introduction to the concepts of ESM and its import-maps, see (e.g.) https://blog.logrocket.com/es-modules-in-browsers-with-import-maps/ (**Reading this #4279 will be easier if you've already been introduced to the concepts.**) * This issue corresponds to [approach 4 in the gist](https://gist.github.com/totten/5c34e3885a4fe7002f990e09395b4294#approaches-to-linkingloading-es6-for-civicrm), except that (now) Firefox and Safari both have native support for import-maps. Part 1. Basic ESM Support -------------------------------- CiviCRM currently allows you to add Javascript to a page. For example: ```php Civi::resources()->addScript('doStuff();'); Civi::resources()->addScriptFile('my_extension', 'js/foo.js'); Civi::resources()->addScriptUrl('https://example.com/bar.js'); ``` generates output like: ```html <script type="text/javascript"> doStuff(); </script> <script type="text/javascript" src="https://example.com/path/to/my_extension/js/foo.js"></script> <script type="text/javascript" src="https://example.com/bar.js"></script> ``` ESM documents are slightly different from regular Javascript documents -- e.g. they support the `import` and `export` keywords. Browsers require a different declaration for ESM: ```html <script type="module"> import doStuff from './do-stuff.js'; doStuff(); </script> <script type="module" src="https://example.com/path/to/my_extension/js/foo.js"></script> <script type="module" src="https://example.com/bar.js"></script> ``` On the PHP side, we probably need a way to flag resources as ESMs. Perhaps: ```php Civi::resources()->addScript('import doStuff from \'./do-stuff.js\'; doStuff();', ['esm' => TRUE]); Civi::resources()->addScriptFile('my_extension', 'js/foo.js', ['esm' => TRUE]); Civi::resources()->addScriptUrl('https://example.com/bar.js', ['esm' => TRUE]); ``` or ```php Civi::resources()->addModule('import doStuff from \'./do-stuff.js\'; doStuff();'); Civi::resources()->addModuleFile('my_extension', 'js/foo.js'); Civi::resources()->addModuleUrl('https://example.com/bar.js'); ``` Other considerations: * Does it matter if or how ESM's appear in `hook_civicrm_coreResourceList`? Assuming we keep our current file-names (`*.js`), then there'd be no way to distinguish vanilla JS files from ESM files. OTOH, that hook was [deprecated circa 5.31](https://docs.civicrm.org/dev/en/latest/hooks/hook_civicrm_coreResourceList/). The newer `hook_civicrm_alterBundle` should be more agreeable (e.g. allowing `$bundle->addModuleFile()`, `$bundle->filter()`, etc). * Is it absolutely necessary to add more methods or flags to `Civi::resources()`? Hypothetically, we could do a global swap in all generated HTML (replace `<script type="text/javascript">` with `<script type="module">`). However, [there are differences of interpretation](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules#other_differences_between_modules_and_standard_scripts) - e.g. wrt scoping and global variables. I did a quick hack to use `type="module"` globally... and the page didn't render. Nothing's impossible, but changing all-at-once feels like a heavier lift. Part 2. Import Map Support ----------------------- Outputting `<script type="module">` is a prerequisite for using ESM's in Civi. However, it's not sufficient - a CiviCRM deployment is split across various folders (*what with core, core-exts, downloaded-ext's, CMS plugins, and so on*). With basic ESM's, it's tricky to reference all these folders. By adding an import-map, we can load dependencies from all these different folders. For example, on a site with `civicrm`, `search_kit`, and `mosaico`, you might generate an import-map that looks like this: ```html <script type="importmap"> { "imports": { "civicrm/": "https://example.com/modules/civicrm/js/", "search_kit/": "https://example.com/modules/civicrm/ext/search_kit/js/" "mosaico/": "https://example.com/files/civicrm/ext/mosaico/js" } } </script> ``` which would enable imports from those paths, as in ```javascript import searchFoo from 'search_kit/foo.js'; import mosaicoFoo from 'mosaico/foo.js'; ``` There are a few important considerations here. In no particular order... * [Quote](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap): "Only the first import map in the document with an inline definition is processed; any additional import maps and external import maps are ignored." -- This means that the importmap will (eventually) be subject to contention among applications like CiviCRM, Drupal, WordPress, etc. We'll (eventually) need a way to integrate the import-maps from different applications. * AFAICS, the CMS's do not currently have a standard way to handle import-maps. But it's on their radar. (Ex: [Drupal issue](https://www.drupal.org/project/drupal/issues/3331393), [WP/Gutenberg issue](https://github.com/WordPress/gutenberg/issues/36716)) We should expect that (eventually) some CMS's will define their own registry. Even when they don't, we should expect *other* CMS plugins will start adding import-maps. * There is a _logical namespace_. We need some guideline for organizing in the logical-namespace. * Example: Map between CiviCRM extension-names and ESM folders (as in the `search_kit` and `mosaico` items above). * Given that every page is limited to one import-map, that means the _logical namespace_ will be shared between Civi extensions and `$open_ended_future_cms_stuff`. We should probably use a prefix. * Should we automatically map all extensions -- or require an opt-in (e.g. `hook_civicrm_esm` or `hook_civicrm_importMaps`)? How confident are we that CiviCRM extensions should be mapped 1-to-1 with ESM's folder namespace? * "The `src`, `async`, ... attributes must not be specified." ([source](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap)) -- The entire importmap must be outputted inline as part of the top-level HTML page; you can load an external map (`<script type="importmap" src="url">`). However, this should eventually change as [WICG's specs define define a way to move the importmap to a `src`](https://github.com/WICG/import-maps#import-map-processing). Given the constraints and expectation of future change, this approach comes to mind: 1. Define `hook_civicrm_esmImportMap(array &$importMap)` as a way for CiviCRM (core/exts) to modify the import-map. 2. Define `mixin/esm@1.0.0`. This is our first-best-guess at how to map between extensions and ESM folders, Maybe: ```php return function ($mixInfo) { Civi::dispatcher()->addListner('hook_civicrm_esmImportMap', function (&$importMap) use ($mixInfo) { $logicalPath = 'civicrm/' . $mixInfo->shortName . '/'; $physicalPath = Civi::resources()->getUrl($mixInfo->longName) . '/'; $importMap[$logicalPath] = $physicalPath; }); } ``` 3. Provide an initial implementation which fires `hook_esmImportMap` and outputs the `<script type="importmap">...</script>` in the page-header. * However, we should expect that (in the future) UF-integrations will need to swap this -- they will instead ask CiviCRM for `$importMap` and pass that to a CMS API. * Additionally, if this mechanism is swappable, then that should improve the prognosis for adding the [es-module-shims](https://github.com/guybedford/es-module-shims) to extend compatibility to older browsers.