DSL fun

One can write an entire book on this topic. I'm just going to explain here how you would write a dead simple DSL and a respective interpreter. Obligatory while I have been paid write interpreters for DSLs in a past position it is not my area of expertise. I am not an expert at this by any means, but it's fun to write-up when it makes sense.

You probably shouldn't do this

When you write a language you are leaving the relatively well traveled path of debugging tools and fairly reproducible output. The moment you write your own language you're likely stepping away from having a clean and highly traceable program because you probably are not building debugging and performance analysis tools for your DSL. You are likely to spend a lot of time looking at the interpreter you wrote asking yourself is the issue my input file or the actual interpreter?

Even if you get everything working keep in mind there are entire languages like Lua that are designed to be embedded in applications. How is your DSL comparing to that option? And even if it is comparable people still need to learn it means adoption is likely to be slowed by that.

When you probably should write a DSL

In my opinion if:

• The language will be simple and easy for people to pickup.
• The language abstracts\automates a lot of tedious work.
• There is not a better language you can embed (even Redis went with just using Lua for this purpose)
• You want to learn

Then yes you should write the language out.

A short explanation\example

What you need end of the day is a way to process the series of inputs from your program and map them into respective functions that you require. For example, you may opt to use %% as the separator for something like variables in a templating system, after which you can simply go through and replace the values.

Or for a function you may add a reserved keyword (common in languages) that allows the user to define an input and then proceed to map each value to the respective language. For example consider the following: var func = (x, y) { print(x + y)}. Here you would use the var as the declaration for your function, assign an internal name of func then convert over the print function to the respective language (or bytecode).

Effectively what you are doing here is your interpreter (of whatever bytecode runtime you are on) is running whatever you converted the input file to. So if you have a file containing a file write you can cleanly convert that over to your own clean\abstracted file write version. For example save.write() in a game DSL might convert to: write the current save state, copy it to a remote server, and reload game state.

So let's say I want a DSL to handle the save state of a game. The user provides a single set of operations. Say a code block like this:

world.create(); let saveFile = world.state(); let playerFile = player.state(); save.multiple(saveFile, playerFile);

Here we would assume the user has been provided global objects for world, player, and save. When the user calls world.create() this is converted to whatever runtime is underlying\being targeted with the respective utility doing things like creating a random seed, writing the file to disk, and initializing the game state. Meanwhile, world.state() and player.state() return whatever object is currently available for the respective states, this is likely a reference to whatever is running within a game engine itself and is likely being dynamically written to as enemies are created\defeated or as the player progresses through the game. We can also make save.multiple() we would have a loop internally which saves the file to an offsite cloud server using the defaults that the game engine has been configured for. To identify that we have a function we can search for ().

Meanwhile, to identify that we have a variable we can search for let and then simply translate the line accordingly. So we would see an array on let saveFile = world.state() that translates to ["let", "saveFile", "=", "world.state()]. We infer from the = this is an assignment and that the key following is the variable name. For the world.state() we subdivide this internally to ["world", "state", ()] and we can infer the first key is the world variable, state is the function because it is followed by (), and because () is empty we can infer there are no arguments. We would then convert the following case: variable of user calls the state function with no arguments into the respective code for our interpreter.

For an example of this embedded into a templating system please take a look at my website write-up and the code.

Gotchas

Understand that some language functions are way easier to implement than others and if you are truly doing a DSL what you probably want is just a few variables along with a handful of functions. For example...consider the complexity required to implement something like reflection that lets you modify the existing objects of a language even if they would otherwise be protected. Now you need to write functionality that allows you to go back into whatever output you have processed and adjust it accordingly at runtime. So you're doing to add a block of code that will modify the existing output while it's running? A lot of complexity for little gain.

Unit tests and integration tests are critical when you are writing a DSL if you want to have any hope of maintaining it long term. Specifically you need to have tests to verify the input and output are exactly what you expect because without this a single change runs the risk of not only breaking one program, but everything built on top of it. This same principle applies to really any major runtime as well, you have to test every use case to ensure there are no regressions between the builds.

Documentation for yourself and anyone you expect to use the DSL in the long term is essential as otherwise you will be guessing what a random function does or even the language syntax.