Seeing code in the config

By: Kannan Ramamoorthy On: Thu 15 October 2020
In: DSL
Tags: #Expressive-Configs #DSL #S-expression

About:

This is an effort to give a perspective to see config(data) as a code, which is not typical for programmers from an imperative programming background. With this you‘ll be able to gain a technique with which you can express any arbitrary logic as part of your config(data). In other words, where others see just data you can see code.

For the benefit of a larger audience, I’m trying to explain this with an example and trying to be as concrete as possible.

Context:

A while back, I was working on the problem of implementing a survey module for one of our clients. This example is about a subset of the bigger problem that any typical Survey module has to solve, the problem of conditional display of questions.

Problem in focus:

Let’s get straight into the problem.

Assume that we are building a survey web-app. Ans assume that we have to implement conditional branching. i.e., certain questions need to be displayed only when certain conditions are met (For ex, display/hide a question based on the answer given by the user for another question). By de facto, consider that we are getting the questions and answers as a JSON and the UI is deciding which questions to show/hide. Let’s see how we can achieve conditional branching here.

In detail,

  • Say you have 3 MCQ questions as part of your survey - Q(1|2|3).

  • And assume each of them has respective answers Q(1|2|3)A(1|2|3).

  • Assume, you have to write a conditional display for Q2, that displays Q2 only when the answer for Q1 is Q1A1.

Solution - first cut:

Based on a conversation with a couple of Java developers, I assume that if you are from a background of imperative language, your solution for conditional display could most likely looks like this,

{
  "questions": [
    {
      "qId": "Q1",
      "options": [
        {
          "id": "Q1A1",
          "displayOnSelect": "Q2"
        },
        {
          "id": "Q1A2"
        },
        {
          "id": "Q1A3"
        }
      ]
    },
    {
      "qId": "Q2",
      "options": [
        {
          "id": "Q2A1"
        },
        {
          "id": "Q2A2"
        },
        {
          "id": "Q2A3"
        }
      ],
      "display": "hidden"
    },
    {
      "qId": "Q3",
      "options": [
        {
          "id": "Q3A1"
        },
        {
          "id": "Q3A2"
        },
        {
          "id": "Q3A3"
        }
      ],
      "display": "hidden"
    }
  ]
}

Where-in, in your code you give the meaning to the json by defining, By default all questions are displayed until "display" is mentioned as hidden. When a particular answer is selected the question mentioned in "displayOnSelect" will be displayed

If you think about alternative choice write it down and see how it evolves as the below complexities are added. And feel free to share it with me, curious to know about it.

Adding complexity:

Think about it how would you address the below scenarios,

  • Display Q3 when the selected answers are Q1A1 and Q2A1.

  • Display Q4 when the selected answers are Q1A1 or Q2A1.

  • Also think about the case in which you get input from the user and use regex match or numerical comparison on that.

You’ll be required to invent a syntax to express the logic and give meaning to those syntax by interpreting it in a certain way as part of your code if you follow the aforementioned approach.

Problem with the solution:

The problem with the above solution is, it is not easily extensible when additional constraints are added, we treat the json just as a data and the whole logic of display is just implemented in the code.

Alternative Solution:

Instead, think if we can have an expression for each field that says whether to display that field or not. And think if the syntax of the expression is extensible to express any logic. And all we have to do in the program is to evaluate that expression and decide.

So do we have to embed a program inside the data and evaluate it as needed? If so, what would be the syntax of the language and how would we evaluate it?

Since we’re dealing with the display of survey questions in the browser, can we embed plain javascript in json and call an eval? Nope, eval doesn’t have a sandbox and could cause security issues. Otherwise it’s not a bad idea.

So how about we come up with a custom syntax for the language that we’re going to use to define the logic of which questions are available? Yeah, we can. And with that we also need to make sure that the bringing in custom language doesn’t pull in a lot of other complexities such as dealing with parse, execution environment, etc.

So the simplest approach that we came up with is,

  • Use S-expression to express the condition.

  • Evaluate the expression using a custom implementer in the context of the answer given.

  • Based on the response of the expression, decide whether to display a particular answer or not.

Solution in action:

Expressing the logic:

Lets see how we can achieve our goal on implementation the conditional display of questions when the conditions get complicated.

For ex, your data for Q3 with the logic “Display Q3 when the selected answers are Q1A1 and Q2A1." can be like this,

{
      "qId": "Q3",
      "canDisplay": ["and", ["equals?", "Q1A1", ["answer", "Q1"]] , ["equals?", "Q2A1", ["answer", "Q2"]]],
      "options": {
        "id": "Q3A1"
      }
}

So what we have did here is, we have expressed the aforementioned condition within our data(json), using s-expression, with the custom operators that are needed.

For example, equals?, answer and and are the operators.

And the meaning of the expression and the sub-expression would be as in the below table,

Expression

Meaning

["answer", "Q1"]

Gets and returns the answer for the question with Q1.

["equals?", "Q1A1", ["answer", "Q1"]

Check if the answer for the question Q1 is "Q1A1". Returns true or false.

["and", ["equals?", "Q1A1", ["answer", "Q1"]] , ["equals?", "Q2A1", ["answer", "Q2"]]]

Check if the answer for the question Q1 is "Q1A1". Returns true or false.

Evaluating the logic:

To evaluate the expression , you can use the library nisp and can extend it with the implementation of required operators(ex, “and", “equals", “answer"). (You can even implement your own quickly. We did a Java variant in just a couple of hours). , which lets us evaluate s-expressions.

For ex, to evaluate the aforementioned expression, your code might looks like this,

import nisp from 'nisp'
var sandbox = {
    'answer': (quid) => answers[quid],
    'and': (e1, e2) => e1 && e2,
    'equals?': (v1, v2) => v1 == v2
};
var exp = ["and", ["equals?", "Q1A1", ["answer", "Q1"]] , ["equals?", "Q2A1", ["answer", "Q2"] ]];
nisp(exp, sandbox); // The return value here is expected to be a boolean. Use the return value to make a choice.

Where the answers is the global that you will be required to maintain, containing an array of answers given by the user.

In this way, you can evaluate expressions to express any logic and it’s completely a sandboxed evaluation.

Advantage:

  • Simple syntax, we can use the existing json parser to parse our custom code.

  • As proven by Lisp family of languages, s-expression can be used to express any arbitrary logic that we need.

  • We can implement an s-expression evaluator in any languages in a couple of hours.

Relevant thoughts:

  • It’s not that this is the design for all the survey module, if your requirement doesn’t need complex conditions as mentioned above, this would be over-engineering.

  • Though the implementation/idea per se is not so difficult, it helps you gain a new perspective to express your logic as part of data in an extensible and elegant way.

  • There are implementations of NISP in different languages hence the same expression can be used to be evaluated in the backend as well if required.

  • For Java developers, using Code in Config is common using SpEL. The advantage of this is, you have precise sandbox control and let your config be shared with applications in other languages as well. Java evaluator for S-expression is available here.

  • There is also a concept of J-expression which has the same nature as s-expression and expresses logic in json instead of deeply nested lists.

  • It’s something worth exploring for the people who come from an imperative programming background. The perspective of seeing code as part of data is something that I should thank Clojure for(and Tamizh who introduced NISP in the first Chennai Clojure meet).
If you found the article helpful, please share or cite the article, and spread the word:

For any feedback or corrections, please write in to: kannangce@rediffmail.com

comments powered by Disqus