durable_rules

for real time analytics (a Ruby, Python and Node.js Rules Engine)

durable_rules is a polyglot micro-framework for real-time, consistent and scalable coordination of events. With durable_rules you can track and analyze information about things that happen (events) by combining data from multiple sources to infer more complicated circumstances.

A full forward chaining implementation (Rete) is used to evaluate facts and events in real time. A simple meta-linguistic abstraction lets you define simple and complex rulesets as well as control flow structures such as flowcharts, statecharts, nested statecharts and time driven flows.

The durable_rules core engine is implemented in C, which enables fast rule evaluation as well as muti-language support.

durable_rules can be scaled out by offloading state to a data store out of process such as Redis. State offloading is extensible, so you can integrate the data store of your choice.

In durable_rules V2, less is more: The Rete tree is fully evaluated in C. Thus, the framework is 5x to 10x faster (depending on the scenario) and does not require Redis. The programming model for posting events, asserting and retracting facts is synchronous and does not prescribe any web framework.

Getting Started

Using your scripting language of choice, simply describe the event to match (antecedent) and the action to take (consequent).

Node.js

To install the framework do: npm install durable

1var d = require('durable');
2
3d.ruleset('test', function() {
4    // antecedent
5    whenAll: m.subject == 'World'
6    // consequent
7    run: console.log('Hello ' + m.subject)
8});
9
10d.post('test', {subject: 'World'});

Python

To install the framework do: pip install durable_rules

1from durable.lang import *
2
3with ruleset('test'):
4    # antecedent
5    @when_all(m.subject == 'World')
6    def say_hello(c):
7        # consequent
8        print ('Hello {0}'.format(c.m.subject))
9
10post('test', { 'subject': 'World' })

Ruby

To install the framework do: gem install durable_rules

1require "durable"
2
3Durable.ruleset :test do
4  # antecedent
5  when_all (m.subject == "World") do
6    # consequent
7    puts "Hello #{m.subject}"
8  end
9end
10
11Durable.post :test, { :subject => "World" }

Forward Inference

durable_rules super-power is the foward-chaining evaluation of rules. In other words, the repeated application of logical modus ponens to a set of facts or observed events to derive a conclusion. The example below shows a set of rules applied to a small knowledge base (set of facts).

Node.js

1var d = require('durable');
2
3d.ruleset('animal', function() {
4    whenAll: {
5        first = m.predicate == 'eats' && m.object == 'flies' 
6        m.predicate == 'lives' && m.object == 'water' && m.subject == first.subject
7    }
8    run: assert({ subject: first.subject, predicate: 'is', object: 'frog' })
9
10    whenAll: {
11        first = m.predicate == 'eats' && m.object == 'flies' 
12        m.predicate == 'lives' && m.object == 'land' && m.subject == first.subject
13    }
14    run: assert({ subject: first.subject, predicate: 'is', object: 'chameleon' })
15
16    whenAll: m.predicate == 'eats' && m.object == 'worms' 
17    run: assert({ subject: m.subject, predicate: 'is', object: 'bird' })
18
19    whenAll: m.predicate == 'is' && m.object == 'frog'
20    run: assert({ subject: m.subject, predicate: 'is', object: 'green' })
21
22    whenAll: m.predicate == 'is' && m.object == 'chameleon'
23    run: assert({ subject: m.subject, predicate: 'is', object: 'green' })
24
25    whenAll: m.predicate == 'is' && m.object == 'bird' 
26    run: assert({ subject: m.subject, predicate: 'is', object: 'black' })
27
28    whenAll: +m.subject
29    run: console.log('fact: ' + m.subject + ' ' + m.predicate + ' ' + m.object)
30});
31
32d.assert('animal', { subject: 'Kermit', predicate: 'eats', object: 'flies'});
33d.assert('animal', { subject: 'Kermit', predicate: 'lives', object: 'water'});
34d.assert('animal', { subject: 'Greedy', predicate: 'eats', object: 'flies'});
35d.assert('animal', { subject: 'Greedy', predicate: 'lives', object: 'land'});
36d.assert('animal', { subject: 'Tweety', predicate: 'eats', object: 'worms'});

Python

1from durable.lang import *
2
3with ruleset('animal'):
4    @when_all(c.first << (m.predicate == 'eats') & (m.object == 'flies'),
5              (m.predicate == 'lives') & (m.object == 'water') & (m.subject == c.first.subject))
6    def frog(c):
7        c.assert_fact({ 'subject': c.first.subject, 'predicate': 'is', 'object': 'frog' })
8
9    @when_all(c.first << (m.predicate == 'eats') & (m.object == 'flies'),
10              (m.predicate == 'lives') & (m.object == 'land') & (m.subject == c.first.subject))
11    def chameleon(c):
12        c.assert_fact({ 'subject': c.first.subject, 'predicate': 'is', 'object': 'chameleon' })
13
14    @when_all((m.predicate == 'eats') & (m.object == 'worms'))
15    def bird(c):
16        c.assert_fact({ 'subject': c.m.subject, 'predicate': 'is', 'object': 'bird' })
17
18    @when_all((m.predicate == 'is') & (m.object == 'frog'))
19    def green(c):
20        c.assert_fact({ 'subject': c.m.subject, 'predicate': 'is', 'object': 'green' })
21
22    @when_all((m.predicate == 'is') & (m.object == 'chameleon'))
23    def grey(c):
24        c.assert_fact({ 'subject': c.m.subject, 'predicate': 'is', 'object': 'grey' })
25
26    @when_all((m.predicate == 'is') & (m.object == 'bird'))
27    def black(c):
28        c.assert_fact({ 'subject': c.m.subject, 'predicate': 'is', 'object': 'black' })
29
30    @when_all(+m.subject)
31    def output(c):
32        print('Fact: {0} {1} {2}'.format(c.m.subject, c.m.predicate, c.m.object))
33
34
35assert_fact('animal', { 'subject': 'Kermit', 'predicate': 'eats', 'object': 'flies' })
36assert_fact('animal', { 'subject': 'Kermit', 'predicate': 'lives', 'object': 'water' })
37assert_fact('animal', { 'subject': 'Greedy', 'predicate': 'eats', 'object': 'flies' })
38assert_fact('animal', { 'subject': 'Greedy', 'predicate': 'lives', 'object': 'land' })
39assert_fact('animal', { 'subject': 'Tweety', 'predicate': 'eats', 'object': 'worms' })        

Ruby

1require "durable"
2
3Durable.ruleset :animal do
4  when_all c.first = (m.predicate == "eats") & (m.object == "flies"),  
5          (m.predicate == "lives") & (m.object == "water") & (m.subject == first.subject) do
6    assert :subject => first.subject, :predicate => "is", :object => "frog"
7  end
8
9  when_all c.first = (m.predicate == "eats") & (m.object == "flies"),  
10          (m.predicate == "lives") & (m.object == "land") & (m.subject == first.subject) do
11    assert :subject => first.subject, :predicate => "is", :object => "chameleon"
12  end
13
14  when_all (m.predicate == "eats") & (m.object == "worms") do
15    assert :subject => m.subject, :predicate => "is", :object => "bird"
16  end
17  
18  when_all (m.predicate == "is") & (m.object == "frog") do
19    assert :subject => m.subject, :predicate => "is", :object => "green"
20  end
21    
22  when_all (m.predicate == "is") & (m.object == "chameleon") do
23    assert :subject => m.subject, :predicate => "is", :object => "green"
24  end
25
26  when_all (m.predicate == "is") & (m.object == "bird") do
27    assert :subject => m.subject, :predicate => "is", :object => "black"
28  end
29    
30  when_all +m.subject do
31    puts "fact: #{m.subject} #{m.predicate} #{m.object}"
32  end
33end
34
35Durable.assert :animal1, { :subject => "Kermit", :predicate => "eats", :object => "flies" }
36Durable.assert :animal1, { :subject => "Kermit", :predicate => "lives", :object => "water" }
37Durable.assert :animal1, { :subject => "Greedy", :predicate => "eats", :object => "flies" }
38Durable.assert :animal1, { :subject => "Greedy", :predicate => "lives", :object => "land" }
39Durable.assert :animal1, { :subject => "Tweety", :predicate => "eats", :object => "worms" }

Pattern Matching

durable_rules provides string pattern matching. Expressions are compiled down to a DFA, guaranteeing linear execution time in the order of single digit nano seconds per character (note: backtracking expressions are not supported).

Node.js

1var d = require('durable');
2
3d.ruleset('test', function() {
4    whenAll: m.subject.matches('3[47][0-9]{13}')
5    run: console.log('Amex detected in ' + m.subject)
6    
7    whenAll: m.subject.matches('4[0-9]{12}([0-9]{3})?')
8    run: console.log('Visa detected in ' + m.subject)
9    
10    whenAll: m.subject.matches('(5[1-5][0-9]{2}|222[1-9]|22[3-9][0-9]|2[3-6][0-9]{2}|2720)[0-9]{12}')
11    run: console.log('Mastercard detected in ' + m.subject)
12});
13
14d.assert('test', { subject: '375678956789765' });
15d.assert('test', { subject: '4345634566789888' });
16d.assert('test', { subject: '2228345634567898' });

Python

1from durable.lang import *
2
3with ruleset('test'):
4    @when_all(m.subject.matches('3[47][0-9]{13}'))
5    def amex(c):
6        print ('Amex detected {0}'.format(c.m.subject))
7
8    @when_all(m.subject.matches('4[0-9]{12}([0-9]{3})?'))
9    def visa(c):
10        print ('Visa detected {0}'.format(c.m.subject))
11
12    @when_all(m.subject.matches('(5[1-5][0-9]{2}|222[1-9]|22[3-9][0-9]|2[3-6][0-9]{2}|2720)[0-9]{12}'))
13    def mastercard(c):
14        print ('Mastercard detected {0}'.format(c.m.subject))
15
16assert_fact('test', { 'subject': '375678956789765' })
17assert_fact('test', { 'subject': '4345634566789888' })
18assert_fact('test', { 'subject': '2228345634567898' })

Ruby

1require "durable"
2Durable.ruleset :test do
3  when_all m.subject.matches('3[47][0-9]{13}') do
4    puts "Amex detected in #{m.subject}"
5  end
6  
7  when_all m.subject.matches('4[0-9]{12}([0-9]{3})?') do
8    puts "Visa detected in #{m.subject}"
9  end
10  
11  when_all m.subject.matches('(5[1-5][0-9]{2}|222[1-9]|22[3-9][0-9]|2[3-6][0-9]{2}|2720)[0-9]{12}') do
12    puts "Mastercard detected in #{m.subject}"
13  end
14end
15
16Durable.assert :test, { :subject => "375678956789765" }
17Durable.assert :test, { :subject => "4345634566789888" }
18Durable.assert :test, { :subject => "2228345634567898" }

Business Rules and Miss Manners

durable_rules can also be used to solve traditional Production Business Rules problems. This example is an industry benchmark. Miss Manners has decided to throw a party. She wants to seat her guests such that adjacent people are of opposite sex and share at least one hobby.

Note how the benchmark flow structure is defined using a statechart to improve code readability without sacrificing performance nor altering the combinatorics required by the benchmark. For 128 guests, 438 facts, the execution time is 450 ms in node.js and 600 ms in Python and Ruby.

IMac, 4GHz i7, 32GB 1600MHz DDR3, 1.12 TB Fusion Drive

• Ruby
• Python
• Node.js

Image recognition and Waltzdb

Waltzdb is a constraint propagation problem for image recognition: given a set of lines in a 2D space, the system needs to interpret the 3D depth of the image. The first part of the algorithm consists of identifying four types of junctions, then labeling the junctions following Huffman-Clowes notation. Pairs of adjacent junctions constraint each other’s edge labeling. So, after choosing the labeling for an initial junction, the second part of the algorithm iterates through the graph, propagating the labeling constraints by removing inconsistent labels.

In this case too, the benchmark flow structure is defined using a statechart to improve code readability. The benchmark requirements are not altered. The execution time, for the case of 4 regions, is 430 ms in node.js, 654 ms in Python and 552 ms in Ruby.

IMac, 4GHz i7, 32GB 1600MHz DDR3, 1.12 TB Fusion Drive

• Node.js
• Ruby
• Python

To Learn More

Reference Manual:

Ruby

Python

Node.js

JSON