How to create a Vite plugin for a custom programming language with Hot Module Reloading

Liana P

Published on July 05, 2023

The Narrat Vite plugin with hot module reloading / hot module replacement

Narrat is the game engine I created. As it has been used more and more over the years, I’ve been consistently improving its development tools. At some point I migrated the build tools for games and narrat itself to using Vite.

More recently, I realised that it would be really useful to have Hot Module Reloading (HMR, also known as Hot Module Replacement, or Hot Code Swapping).

With this, when people are working on games, they can see their changes immediately, without having to reload the page. This is particularly useful to make changes in the middle of a complex playthrough, where reloading the page and playing all the way back to getting the same state can be a pain.

How it’s made

Looking at the Vite plugins API told me that it’s mostly like Rollup plugins, but with some features on top.

Plugins to transform files with an extension

Like in their example, writing a plugin that can transpile files is a matter of returning an object with a transform function:

const fileRegex = /\.(my-file-ext)$/

export default function myPlugin() {
  return {
    name: 'transform-file',

    transform(src, id) {
      if (fileRegex.test(id)) {
        return {
          code: compileFileToJS(src),
          map: null, // provide source map if available
        }
      }
    },
  }
}

The transform function receives the source code of the file, and the file’s path. It returns an object with the transpiled code, and optionally a source map.

In the case of Narrat, the engine takes care of “compiling” the scripts, so I really only need this transformation plugin to return a string with the narrat script. A first version looked a bit like that:

const fileRegex = /\.(narrat|nar)$/;

function narratPlugin() {
  return {
    name: 'narrat',

    transform(src, id) {
      if (fileRegex.test(id)) {
        const fileName = id.match(/[^/\\]*$/)[0];
        return {
          code: `
const narratCode = ${JSON.stringify(src)};

export default narratCode;`,
          map: null, // provide source map if available
        };
      }
    },
  };
}

module.exports = narratPlugin;

With this, the vite config can import the plugin and use it, and then the actual scripts can be imported as if they were regular JS files:

in vite.config.ts:

import Narrat from './plugins/vite-plugin-narrat';
// ...
export default defineConfig({
  // ...
  plugins: [
    // ...
    Narrat(),
  ],
  // ...
});

Then for example in src/main.ts:

import narratCode from './narrat/main.narrat';

console.log(narratCode);

At this point we have our narrat files actually imported and available in variables.

Using it in code

I won’t go into details on this part as it’s not Vite related, but essentially I refactored the engine a bit to be able to pass an array of narrat scripts to the app start, which were the files created by the plugin. I also modified them to have a bit more info, so my plugin returned this:

const narratCode = ${JSON.stringify(src)};

export default {
  code: narratCode,
  fileName: '${fileName}',
  id: '${id}',
};

This way, we can easily access the file name, the full path, or the code.

Then, it’s up to individual games to import their various narrat files, put them in an array, and pass them to the engine when the game starts.

TypeScript support for custom extension

One problem that quickly arises is that TypeScript doesn’t know about the .narrat extension, so it will complain when trying to import the files. To fix this, we can create a shim file, inspired by the one Vue.js uses to support .vue files:

In my case, in src/types/app-types I have this interface that defines the content of a narrat file:

export type NarratScript = {
  code: string;
  fileName: string;
  id: string;
};

Then in src/types/narrat-shim.d.ts I have this:

// shims-narrat.d.ts

declare module '*.narrat' {
  import { NarratScript } from './app-types';
  const narratScript: NarratScript;
  export default narratScript;
}

This tells typescript that .narrat files export a NarratScript object.

All that’s needed then is to include this shim in narrat projects (I have added it to the narrat template so newly created games include it).

Hot Module Reloading

The last part is hot module reloading, which is really the actual reason why I did all this. Turns out it’s not that complicated.

The HMR API provides an import.meta.hot value to modules, and more importantly a import.meta.hot.accept function that modules can call with a callback that will be called when the module is updated.

Using this callback, I can get the new value of the file, and call a function within narrat that will take care of re-parsing the script, and replacing all the labels (more on that later).

This is how the plugin looks with hot module reloading:

const fileRegex = /\.(narrat|nar)$/;

function narratPlugin() {
  return {
    name: 'narrat',

    transform(src, id) {
      if (fileRegex.test(id)) {
        const fileName = id.match(/[^/\\]*$/)[0];
        return {
          code: `
const narratCode = ${JSON.stringify(src)};

const HMREventHandler = (newModule) => {
  console.log('Received HMR update for ' + '${id}');
  const narrat = window.narrat
  if (narrat) {
    narrat.handleHMR(newModule);
  }
  const event = new CustomEvent('narrat-hmr', {
    detail: {
      newModule,
    },
  });
  document.body.dispatchEvent(event);
}
if (import.meta.hot) {
  console.log('Accepting HMR update to ' + '${id}');
  import.meta.hot.accept(HMREventHandler);
}
export default {
  code: narratCode,
  fileName: '${fileName}',
  id: '${id}',
};
`,
          map: null, // provide source map if available
        };
      }
    },
  };
}

module.exports = narratPlugin;

The HMREventHandler function is called when the module is updated, and it calls a function in narrat that will take care of updating the script. It also dispatches a custom event that can be listened to by the game, to do any other updates that might be needed.

In this case, for simplicity, I made narrat add a function to the window narrat object, which is generally used to plug into external things. The hot module reload script calls that function with the new module, and it’s up to the engine to handle it.

Handling the hot module reload

Now that we are receiving module updates and notifying the engine of them, we need to handle them. This is the code I wrote for hmr updates:

import { NarratScript } from '@/types/app-types';
import { vm } from '@/vm/vm';
import { ModuleNamespace } from 'vite/types/hot';

export function handleHMR(newModule: ModuleNamespace | undefined) {
  if (!newModule || !newModule.default) {
    return;
  }
  const scriptModule = newModule.default;
  if (isNarratScript(scriptModule)) {
    vm.addNarratScript(scriptModule);
  }
}

export function isNarratScript(
  scriptModule: any,
): scriptModule is NarratScript {
  return (
    typeof scriptModule === 'object' &&
    scriptModule !== null &&
    typeof scriptModule.code === 'string' &&
    typeof scriptModule.fileName === 'string' &&
    typeof scriptModule.id === 'string'
  );
}

The isNarratScript function is a type guard to make sure that the module being loaded is definitely a narrat module conforming to the format we expect. Then, once the handleHMR function has checked that everything about the module seems correct, it can call vm.addNarratScript, which is the internal engine function to add new scripts.

Updating the engine

Updating narrat scripts at this point is pretty easy. Narrat scripts are made of a list of labels (which are basically narrat functions). All labels from every narrat scripts are parsed and added to an object that contains all of them.

This means, reading an updated narrat file is a matter of re-parsing all the labels in it, and overriding them in the object where all the labels are stored.

Once that’s done, the next time the game’s script reaches the label, it will execute the new code.

Result

With all those pieces put together, it all works. Here’s a video demonstrating a narrat game changing live with hot module reloading:

Full code

If you want to see the wider context of the code involved in making this change in Narrat, here’s the link to the HMR pull request.

Articles by Liana P

Writing about things