A better CFengine syntax checker

Within SARA CFengine is used to manage many systems.  I have used CFengine from the early days (version 1 and 2).  CFengine version 3 is a huge improvement over version 1 and 2. The syntax language is better and i really like the the reuse of code and the usage of methods to hide the complexity of a bundle.  A small list on which CFengine is installed:

• LISA Cluster

• GPU Cluster

• HPC Cloud

What i don't like is the way how CFengine reports errors in the syntax, it is cryptic and confusing. The syntax checker can report the wrong line number,  the wrong file or report no error. It can also do a better job of what is being expected.  My opinion is that if the syntax checker does a better job it will lower the learning curve for a beginner and  for experience user it will lower the time to find the syntax error.

CFengine  is now hosted on github so it easy to fork and work on a better syntax checker.  I decided to do it and spent some time on improving the parser,  thanks to SARA for allowing me to start this project.  The most important files for syntax checker are:

1. The lexer (lex): cf3lex.l
2. The parser (yacc): cf3parse.y

My goal is/was that the impact of my changes must be minimal and be concentrated in those 2 files.  The  CFengine syntax checker does not use all the features of lex/yacc to report syntax errors.  I have used some of these lex/yacc features for better syntax error reporting.  The syntax errors are immediately detected by lex/yacc and not after the parsing by C-functions.  

The result is open to discussions/suggestions/improvements and is available at :

• github.com/basvandervlies/core/tree/parser
• github.com/cfengine/core/pull/326

In the process of rewriting the syntax check i have one important suggestion:

1. CFengine has now reserved keywords. My suggestions is to have reserved keywords for e.g. bundle, body, agent, common, .... Most languages have this, e.g. C  has reserved keywords if, else, for, ... 

Examples

Here are some examples that show the difference between my parser and CFengine parser.  I made some syntax errors in a tomcat6.cf bundle.  The CFengine configuration have many bundles that are included in the SARA setup. This will illustrate how CFengine reports syntax errors.

unknown bundle type

bundle comon tomcat6

{

    vars:

        debian_squeeze|debian_lenny|debian_5|debian_6::

            "config_dir"        string => "/etc/tomcat6";

            "restart"           string => "/etc/init.d/tomcat6 restart";

            "webapps_dir"       string => "/var/lib/tomcat6/webapps";

            "log_dir"           string => "/var/log/tomcat6";

            "log_file"          string => "/var/log/tomcat6/catalina.out";

} 

CFengine3 parser:

Reports no error! 

09:56 bas.sara.nl:/var/tmp/bas/cfengine

root# cf-promises -f ./cf-config/promises.cf 

My parser:

10:00 bas.sara.nl:/var/tmp/bas/cfengine

root# own/core/src/cf-promises -f ./cf-config/promises.cf 

filename: ./cf-config/bundles/tomcat6.cf line 19: token: 'comon'

bundle comon tomcat6

        ^ error

error: Unknown bundle type

forgot ';'

bundle common tomcat6

{
    vars:

        debian_squeeze|debian_lenny|debian_5|debian_6::

            "config_dir"        string => "/etc/tomcat6";
            "restart"           string => "/etc/init.d/tomcat6 restart";
            "webapps_dir"       string => "/var/lib/tomcat6/webapps";
            "log_dir"           string => "/var/log/tomcat6"
            "log_file"          string => "/var/log/tomcat6/catalina.out";
}

CFengine3 parser  (it truncates the lines):


16:46 bas.sara.nl:/var/tmp/bas/cfengine
root# cf-promises -f ./cf-config/promises.cf 
cf3> ./cf-config/bundles/tomcat6.cf:29,22: syntax error, near token 'log_file"'
cf3> ./cf-config/bundles/torque.cf:1,17: syntax error, near token 'string'
cf3> ./cf-config/bundles/torque.cf:1,20: syntax error, near token '=>'
cf3> ./cf-config/bundles/torque.cf:1,52: syntax error, near token '/var/log/tomcat6/cat'
cf3> ./cf-config/bundles/torque.cf:1,53: syntax error, near token ';'
cf3> ./cf-config/bundles/torque.cf:2,1: syntax error, near token '}'

My parser:


16:47 bas.sara.nl:/var/tmp/bas/cfengine
root# own/core/src/cf-promises -f ./cf-config/promises.cf 

filename: ./cf-config/bundles/tomcat6.cf line 29: token: '"log_file"'

            "log_file"          string => "/var/log/tomcat6/catalina.out";
             ^ error
error: check previous statement, expected ';'

wrong definition in vars

bundle common tomcat6

{

    vars:

        debian_squeeze|debian_lenny|debian_5|debian_6::

            "config_dir"        string => "/etc/tomcat6";

            "restart"           string => "/etc/init.d/tomcat6 restart";

            "webapps_dir"       string => "/var/lib/tomcat6/webapps";

            "log_dir"           sring => "/var/log/tomcat6";

            "log_file"          string => "/var/log/tomcat6/catalina.out";

}

CFengine3 parser result:

16:59 bas.sara.nl:/var/tmp/bas/cfengine

root# cf-promises -f ./cf-config/promises.cf 

cf3> ./cf-config/bundles/tomcat6.cf:28,59: Constraint lvalue 'sring' is not allowed in bundle category 'vars', near token '/var/log/tomcat6"'

My  parser:

filename: ./cf-config/bundles/tomcat6.cf line 28: token: 'sring'

            "log_dir"           sring => "/var/log/tomcat6";

                                 ^ error

error: 'sring' is not allowed as promise attribute for promise type: 'vars'

Forgot to close a bundle

bundle common tomcat6

{

    vars:

        debian_squeeze|debian_lenny|debian_5|debian_6::

            "config_dir"        string => "/etc/tomcat6";

            "restart"           string => "/etc/init.d/tomcat6 restart";

            "webapps_dir"       string => "/var/lib/tomcat6/webapps";

            "log_dir"           string => "/var/log/tomcat6";

            "log_file"          string => "/var/log/tomcat6/catalina.out";

CFengine3 parser:

09:23 bas.sara.nl:/var/tmp/bas/cfengine

root# cf-promises -f ./cf-config/promises.cf 

cf3> ./cf-config/bundles/tomcat6.cf:32,6: syntax error, near token 'bundle'

cf3> ./cf-config/bundles/torque.cf:1,7: syntax error, near token 'agent'

cf3> ./cf-config/bundles/torque.cf:1,31: syntax error, near token 'tomcat6_mail_log_err'

cf3> ./cf-config/bundles/torque.cf:1,32: syntax error, near token '('

cf3> ./cf-config/bundles/torque.cf:1,42: syntax error, near token 'mail_addrs'

cf3> ./cf-config/bundles/torque.cf:1,43: syntax error, near token ')'

cf3> ./cf-config/bundles/torque.cf:2,1: syntax error, near token '{'

cf3> ./cf-config/bundles/torque.cf:3,12: syntax error, near token 'classes'

cf3> ./cf-config/bundles/torque.cf:5,31: syntax error, near token 'tomcat_not_empty_fil'

cf3> ./cf-config/bundles/torque.cf:5,43: syntax error, near token 'expression'

cf3> ./cf-config/bundles/torque.cf:5,46: syntax error, near token '=>'

Fatal CFEngine error: Too many errors

My parser:

09:29 baa.sara.nl:/var/tmp/bas/cfengine

root# own/core/src/cf-promises -f ./cf-config/promises.cf 

filename: ./cf-config/bundles/tomcat6.cf line 32: token: 'bundle'

bundle agent tomcat6_mail_log_errors(mail_addrs)

 ^ error

error: Bundle body error, expected '}'