[ANN] Grules - DSL for data preprocessing

[Vitalii Fedorenko]

I would like to announce a new project called Grules that I have worked on for two years. It is a Groovy DSL for data preprocessing, i.e. validation and canonicalization.

Validation of input parameters is usually defined using IF statements and exceptions. With a lot of parameters such code becomes less readable and frequently is split between different methods and classes. Furthermore, when a new parameter appears it is easy to forget about its validation, escaping of special symbols, string trimming, etc. To solve these and other problems with input validation, I wrote a Groovy library that allows to define data preprocessing rules via DSL. For example, the following script handles six input parameters: email, login, gender, terms and conditions checkbox, weight, and end date:

// isEmail is a Groovy/Java method that takes an email value as its parameter
email isEmail ["Invalid email"]

// invalidLoginErr and dupLoginErr are String error messages
login isLogin [invalidLoginErr] >> isUnique [dupLoginErr]

// Gender is a Groovy enumeration
gender toEnum(Gender)

// agreeToTerms is a message from a resource bundle
termsCondition[""] !isEmpty [m.agreeToTerms]

// you can use closures as well
weight toPositiveBigDecimal [decimalErr] >> {round(it / 1000)}

// Grules supports logical operators  
endDate isAfterNow && isBefore(deadline) && {it.day != 1}

As the result of this script one will get a map of objects with input values divided in 5 groups:
- valid
- invalid
- missings
- for which a preprocessing rule was not found
- with missing or invalid values dependencies

To include support of the DSL into your project, just add grules.jar to classpath (for example via Maven), and add Groovy scripts that contain validation rules for your data (file names of the scripts must end with Grules).

If you want to try the project, there is Grules console, a demo project in Grails and a hello world project on Github. The hello world project consist of two main files:

HelloGrules.groovy:

package test

email isEmail ["Invalid email"]
age toPositiveInt ["Invalid age"] >> {it > 18} ["You must be adult"]

Test.groovy:

package test

import org.grules.Grules

class Test {

  public static void main(String[] s) {
    def grules = new Grules()
    def result = grules.applyRules(HelloGrules, [email: "megmail.com", age: "35"])
    assert result.cleanParameters.age == 35
    assert "email" in result.invalidParameters
    assert result.invalidParameters.email.errorId == "Invalid email"
    println result
  }
}

To run Hello world, execute ./gradlew in grulesHelloWorld directory.

Please note that this snippet shows only basic features, and you can find much more on Wiki (https://github.com/zhaber/grules/wiki) and in this paper digitalcommons.mcmaster.ca/cgi/viewcontent.cgi?article=8244&context=opendissertations (some of the features described the document are not implemented yet, but core functionality is ready).
Source code: https://github.com/zhaber/grules
Short presentation that explains the motivation for this project: http://www.youtube.com/watch?v=6RYbDRY6cvQ (I make similar comparison of code “noisiness” to what Guillaume Laforge presented at SpringOne 2GX: http://www.infoq.com/presentations/Design-Your-Own-DSL-with-Groovy)
Video with a simple demo project: http://www.youtube.com/watch?v=nPnpWkZ1QAQ

Russian translation: http://habrahabr.ru/post/160085/

Vitalii

[Guillaume Laforge-4]

Interesting, thanks for sharing!

On Sat, Jan 26, 2013 at 1:37 PM, Vitalii Fedorenko <[hidden email]> wrote:

I would like to announce a new project called Grules that I have worked on
for two years. It is a Groovy DSL for data preprocessing, i.e. validation
and canonicalization.

Validation of input parameters is usually defined using IF statements and
exceptions. With a lot of parameters such code becomes less readable and
frequently is split between different methods and classes. Furthermore, when
a new parameter appears it is easy to forget about its validation, escaping
of special symbols, string trimming, etc. To solve these and other problems
with input validation, I wrote a Groovy library that allows to define data
preprocessing rules via DSL. For example, the following script handles six
input parameters: email, login, gender, terms and conditions checkbox,
weight, and end date:

// isEmail is a Groovy/Java method that takes an email value as its
parameter
email isEmail ["Invalid email"]

// invalidLoginErr and dupLoginErr are String error messages
login isLogin [invalidLoginErr] >> isUnique [dupLoginErr]

// Gender is a Groovy enumeration
gender toEnum(Gender)

// agreeToTerms is a message from a resource bundle
termsCondition[""] !isEmpty [m.agreeToTerms]

// you can use closures as well
weight toPositiveBigDecimal [decimalErr] >> {round(it / 1000)}

// Grules supports logical operators
endDate isAfterNow && isBefore(deadline) && {it.day != 1}

As the result of this script one will get a map of objects with input values
divided in 5 groups:
- valid
- invalid
- missings
- for which a preprocessing rule was not found
- with missing or invalid values dependencies

To include support of the DSL into your project, just add grules.jar to
classpath (for example via  Maven
<http://search.maven.org/#artifactdetails%7Corg.grules%7Cgrules%7C0.2.0.8%7Cjar>
), and add Groovy scripts that contain validation rules for your data (file
names of the scripts must end with Grules).

If you want to try the project, there is  Grules console
<http://grules-console.appspot.com>  , a  demo project
<http://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fzhaber%2Fgrules%2Ftree%2Fmaster%2FgrulesDemo&sa=D&sntz=1&usg=AFQjCNFs476lWq9dhDnKekDay_Q9QltNbg>
in Grails and a  hello world
<https://github.com/downloads/zhaber/grules/grules_hello_world.zip>
project on Github. The hello world project consist of two main files:

HelloGrules.groovy:

package test

email isEmail ["Invalid email"]
age toPositiveInt ["Invalid age"] >> {it > 18} ["You must be adult"]

Test.groovy:

package test

import org.grules.Grules

class Test {

  public static void main(String[] s) {
    def grules = new Grules()
    def result = grules.applyRules(HelloGrules, [email: "megmail.com", age:
"35"])
    assert result.cleanParameters.age == 35
    assert "email" in result.invalidParameters
    assert result.invalidParameters.email.errorId == "Invalid email"
    println result
  }
}

To run Hello world, execute ./gradlew in grulesHelloWorld directory.

Please note that this snippet shows only basic features, and you can find
much more on Wiki (https://github.com/zhaber/grules/wiki) and in this paper
digitalcommons.mcmaster.ca/cgi/viewcontent.cgi?article=8244&context=opendissertations
<http://digitalcommons.mcmaster.ca/cgi/viewcontent.cgi?article=8244&context=opendissertations>
(some of the features described the document are not implemented yet, but
core functionality is ready).
Source code: https://github.com/zhaber/grules
Short presentation that explains the motivation for this project:
http://www.youtube.com/watch?v=6RYbDRY6cvQ (I make similar comparison of
code “noisiness” to what Guillaume Laforge presented at SpringOne 2GX:
http://www.infoq.com/presentations/Design-Your-Own-DSL-with-Groovy)
Video with a simple demo project: http://www.youtube.com/watch?v=nPnpWkZ1QAQ

Russian translation: http://habrahabr.ru/post/160085/

Vitalii



--
View this message in context: http://groovy.329449.n5.nabble.com/Grules-DSL-for-data-preprocessing-tp5712613.html
Sent from the groovy - user mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one