Query language

Algorithms Library Toolkit is a ….

aql in general

The command line interface aql2 binary allows interaction with all algorithms through an interactive command line interface. The syntax of the language is similar to shell known from Unix operating systems. Internally it handles the passing of algorithm’s results to other algorithm’s parameters directly without any transformation or manipulation. Such an approach allows the fastest interpretation of the algorithm interconnection. Even though the command line interface tries to hide some implementation details, a user can feel the fact that the implementation language of the library is C++ in some cases. For example, the command line interface is aware of datatypes and it is also unable to handle templates (used to design datatypes and algorithms) otherwise than statically. This, however, does not influence the overall functionality.

The command line interface binary at first automatically loads existing datatypes and algorithm provided by the linked libraries. The command line interface also provides a load and an unload command to load and unload an arbitrary library which can contribute to currently available datatypes and algorithms.

The command line interface is planned to be extended to support some procedural like language which may cause a drop of backward compatibility.

execution parameters

The query language shell binary accepts the following parameters:

• -c – allows execution of a single command
• -f – executes commands from a file
• -e – sets an enviroment variable
• -i – open interactive mode

The default behavior of the shell binary is to open the interactive mode. The default behavior is disabled if any of -c or -f is specifies. The shell binary will stay in interactive mode if -i is used with -c or -f parameter.

The -c, -f and -i parameters are executed in order of appearance. The environment variables are set up before any command is executed.

Any quit command executed from parameter, a file, or from interactive shell stops the complete sequence of execution plan.

query language specification

The basics of the command line interface language is similar to bash, featuring subqueries and pipes. However, the language is extended with declaration of variables, functions, and procedures. It also features sequencing of commands with semicolons and usual expression syntax. The language is described by syntax similar to EBNF. The grammar’s initial symbol is ‘parse’. The terminal symbols are in uppercase, nonterminals in lowercase. The terminal symbols starting with KW_ prefix represent nonreserved keywords of values equal to what is after the underscore. INTEGER is a sequence of digits; IDENTIFIER is a letter followed by letters, digits and some special symbols; STRING is double-quoted character sequence; FILE represents a file path; TYPE represents a C++ datatype. The terminal symbol end is a special terminal symbol representing the end of the input.

arg
	:	HASH_SIGN ( INTEGER | IDENTIFIER )
	|	IDENTIFIER
	;

template_arg
	:	AT_SIGN arg
	;

file
	:	HASH_SIGN ( INTEGER | IDENTIFIER )
	|	STRING
	|	FILE
	;

type
	:	STRING
	|	TYPE
	;

in_redirect
	:	LEFT_PAREN statement_list RIGHT_PAREN
	|	( LEFT_BRACKET arg RIGHT_BRACKET )?
		( COLON_SIGN ( INTEGER | IDENTIFIER ) )?
		file
	;

out_redirect
	:	DOLAR_SIGN arg
	|	( LEFT_BRACKET arg RIGHT_BRACKET )? file
	;

common
	:	DOLAR_SIGN arg
	|	LESS_SIGN in_redirect
	|	STRING
	|	INTEGER
	|	HASH_SIGN ( INTEGER | IDENTIFIER )
	|	LEFT_BRACE ( param ) * RIGHT_BRACE
	;

param
	:	common
	|	DASH_SIGN
	|	IDENTIFIER
	|	LEFT_PAREN type RIGHT_PAREN param
	;

statement
	:	common ( );
	|	IDENTIFIER template_arg ( COLON_SIGN ( INTEGER | IDENTIFIER ) )? ( param )*
	|	LEFT_PAREN type RIGHT_PAREN statement
	;

statement_list
	:	statement ( PIPE_SIGN statement | MORE_SIGN out_redirect )*
	;

block
	:	BEGIN semicolon_command END
	;

semicolon_command
	:	(BEGIN | IF | WHILE) => command
	|	command SEMICOLON
	;

batch
	:	statement_list
	;

expression
	:	assign_expression
	;

introspect_cast_from_to
	:	( COLON_SIGN ( KW_FROM | KW_TO ) )*
	;

introspect_command
	:	KW_ALGORITHMS arg?
	|	KW_OVERLOADS arg template_arg*
	|	KW_OPERATORS
	|	KW_DATATYPES arg?
	|	KW_CASTS introspect_cast_from_to arg?
	|	KW_NORMALIZATIONS
	|	KW_DENORMALIZATIONS
	|	KW_VARIABLES ( DOLAR_SIGN arg )?
	|	KW_BINDINGS ( HASH_SIGN ( INTEGER | IDENTIFIER ) )?
	;

expression_or_batch
	:	KW_BATCH batch
	|	KW_EXPRESSION? expression
	;

batch_or_expression
	:	KW_EXPRESSION expression
	|	KW_BATCH? batch
	;

qualifiedType
	:	KW_CONST? arg ( AND_OPERATOR | AMPERSAND_SIGN )?
	;

runnableParam
	:	qualifiedType DOLAR_SIGN arg
	;

command
	:	KW_EXECUTE batch_or_expression
	|	KW_PRINT batch_or_expression
	|	KW_QUIT batch_or_expression?
	|	KW_EXIT batch_or_expression?
	|	KW_RETURN batch_or_expression?
	|	KW_HELP arg?
	|	KW_INTROSPECT introspect_command
	|	KW_SET ( UNSIGNED | IDENTIFIER ) ( UNSIGNED | IDENTIFIER | STRING )
	|	KW_SHOW ( UNSIGNED | IDENTIFIER ) KW_MEASUREMENTS KW_AS ( KW_LIST | KW_TREE )
	|	KW_CLEAR KW_MEASUREMENTS
	|	KW_START ( KW_ROOT | KW_OVERALL | KW_INIT | KW_FINALIZE | KW_MAIN | KW_AUXILIARY | KW_PREPROCESS | KW_ALGORITHM ) KW_MEASUREMENT KW_FRAME ( UNSIGNED | IDENTIFIER )
	|	KW_STOP KW_MEASUREMENT KW_FRAME
	|	KW_LOAD ( FILE | STRING )
	|	KW_UNLOAD ( FILE | STRING )
	|	KW_CALC expression
	|	block
	|	KW_EVAL ( UNSIGNED | IDENTIFIER | STRING );
	|	KW_INTERPRET ( FILE | STRING )
	|	KW_IF LEFT_PAREN expression_or_batch RIGHT_PAREN KW_THEN semicolon_command ( KW_ELSE semicolon_command )
	|	KW_WHILE LEFT_PAREN expression_or_batch RIGHT_PAREN KW_DO semicolon_command
	|	KW_BREAK
	|	KW_CONTINUE
	|	KW_DECLARE qualified_type DOLAR_SIGN arg ASSIGN_OPERATOR expression
	|	KW_PROCEDURE ( UNSIGNED | IDENTIFIER | STRING ) LEFT_PAREN ( runnable_param ( COMMA runnable_param )* ) RIGHT_PAREN command
	|	KW_FUNCTION ( UNSIGNED | IDENTIFIER | STRING ) LEFT_PAREN ( runnable_param ( COMMA runnable_param )* ) RIGHT_PAREN RETURNING qualified_type command
	;

parse
	:	command ( SEMICOLON command ) END
	|	END
	;

assign_expression
	:	or_expression ( ASSIGN_OPERATOR assign_expression )?
	;

or_expression
	:	and_expression ( OR_OPERATOR and_expression )*
	;

and_expression
	:	bitwise_or_expression ( AND_OPERATOR bitwise_or_expression )*
	;

bitwise_or_expression
	:	bitwise_and_expression ( PIPE_SIGN bitwise_and_expression )*
	;

bitwise_and_expression
	:	bitwise_xor_expression ( AMPERSAND_SIGN bitwise_xor_expression )*
	;

bitwise_xor_expression
	:	equality_expression ( CARET_SIGN equality_expression )*
	;

equality_expression
	:	relational_expression ( EQUAL_OPERATOR relational_expression | NOT_EQUAL_OPERATOR relational_expression )
	;

relational_expression
	:	add_expression ( LESS_SIGN add_expression | LESS_OR_EQUAL_OPERATOR add_expression | MORE_SIGN add_expression | MORE_OR_EQUAL_OPERATOR add_expression )
	;

add_expression
	:	mul_expression ( PLUS_SIGN mul_expression | MINUS_SIGN mul_expression )*
	;

mul_expression
	:	prefix_expression ( STAR_SIGN prefix_expression | PERCENTAGE_SIGN prefix_expression | SLASH_SIGN prefix_expression )*
	;

prefix_expression
	:	KW_CAST LEFT_PAREN type RIGHT_PAREN
	|	PLUS_SIGN prefix_expression
	|	MINUS_SIGN prefix_expression
	|	EXCLAMATION_MARK prefix_expression
	|	TYLDE_SIGN prefix_expression
	|	INC_OPERATOR prefix_expression
	|	DEC_OPERATOR prefix_expression
	|	suffix_expression
	;

suffix_expression
	:	atom
		(	DOT IDENTIFIER
		|	INC_OPERATOR
		|	DEC_OPERATOR
		)
	;

atom
	:	(KW_TYPE | KW_ACTUAL_TYPE) LEFT_PAREN expression RIGHT_PAREN
	|	(KW_DECLARED_TYPE) LEFT_PAREN expression RIGHT_PAREN¨
	|	IDENTIFIER bracketed_expression_list
	|	DOLAR_SIGN IDENTIFIER
	|	LEFT_PAREN expression RIGHT_PAREN
	|	STRING
	|	UNSIGNED
	;

bracketed_expression_list
	:	LEFT_PAREN expression ( COMMA expression )* RIGHT_PAREN
	;

Introspection

The query language supports the listing of registered algorithms, their overloads, datatypes, and casts. The introspection command is executed by ‘introspect’ keyword followed by the introspected entity, i.e. algorithms, overloads, datatypes, and casts.

Algorithms introspection may use namespace names to narrow the query. The namespace name must end with four-dot. The result of the introspection is a list of all algorithms (filtered by namespace if specified) one per line.

Overloads introspection requires exact algorithm name. The result is a list of signatures of available overloads, one per line. Each signature consists of the return type followed by parameter types.

Datatypes introspection allows printing datatypes that can be exported or imported as an XML file.

Casts introspection prints all available casts. Each cast is represented by to type and from type. The query may be limited to casts from or to the specific type.

An introspection command can list variables available in the environment of the command line interface. The type of variable can be also introspected by suffixing the variable name after introspect variables command.

Environment bindings are handled in the introspection similarly to variables.

Some examples of introspection commands follow.

introspect algorithms
introspect algorithms automaton::
introspect overloads automaton::determinize::Determinize
introspect datatypes
introspect casts
introspect casts :from int
introspect casts :to int
introspect variables
introspect bindings

Execute and print

The execute command is intended to execute a sequence of statemets. Statements are vertical bar separated algorithms introduced by the name, casts to specified type, or one can use a variable, file, string or integer constant, binded value, or set constructor. (The set constructor will in the future acompanied with constructor of other common structures.)

The syntax of execute comand is similar to the syntax of bash. In the simple case the sequence of statements is simply a sequence of algorithms and their parameters.

> print string::Parse @Automaton "NFA a b
>0 0 1
 1 - 2
<2 - -" | Determinize -

Result of a execute command will be printed to the console by default, but it can also be printed to a file and later read.

> print string::Parse @Automaton "NFA a b
>0 0 1
 1 - 2
<2 - -" | Determinize - > automaton.xml
> execute < automaton.xml | Minimize -

File reading is parametrized by some clauses, the argument inside the brackets is a specification of the file type, the argument after the colon is the specification of the content of the file, and arguments after at sign give template parameters is needed.

The file type specifications are ’xml’, ’string’, ’raw’, and ’file’. The ’xml’ file type contains XML representation of some datatype. The ’string’ file type represents string representation of an automaton, grammar, regexp, or some other type (available to be listed via introspection. The ’raw’ file type is a representation of an unranked tree, i.e. XML or linear string, i.e. sequence of chars. The file type reads the content of the file as a standard string.

The string and raw types require the content specification to be set with type argument introduced by a colon. The template parameter is extra information not used now.

> execute "abc" | cli::builtin::WriteFile "file.txt" -
> execute < [ raw ] :string::LinearString file.txt

The use of the in_redirect symbol is overloaded and allows the creation of subquery.

> print Determinize <( string::Parse @Automaton "NFA a b
>0 0 1
 1 - 2
<2 - -")

Out_redirect knows the type of the value printed, hence only the file type specification is required. The syntax is again an identifier in brackets. Based on the specified file type the argument of out_redirect can be limited to certain types only.

> print Determinize <( string::Parse @Automaton "NFA a b
>0 0 1
 1 - 2
<2 - -" ) > [ string ] dfa.txt

The out_redirect is also overloaded to set up a variable. That can later be used as a parameter of an algorithm.

> print Determinize <( string::Parse @Automaton "NFA a b
>0 0 1
 1 - 2
<2 - -" ) > $dfa
> execute Minimize $dfa

The command line interface can be exited with quit command that can also report exit status of the execution. The syntax of the quit is same as execute in that case, however only integer and boolean results are interpreted as exit status. Integer result is used directly, boolean value true is interpretted as 0 exit status and false as 1. The behavior matches the expected results of commands in unix like systems.

Execution parameters

test.aql:

> execute string::Parse @Automaton "
DFA 0 1
<>0 0 0" > $dfa
> print "f1"
> print "f2"

$ aql2 -c "print 1" -c "print 2" -f test.aql  ; echo "retval -> $?"
1
2
f1
f2
retval -> 0

$ aql2 -c "print 1" -c "print 2" -f test.aql -i  ; echo "retval -> $?"
1
2
f1
f2
> print "i"
i
> <EOF>
retval -> 0

$ aql2 -c "print 1" -c "quit 55" -f test.aql -i  ; echo "retval -> $?"
1
retval -> 55

Measurements in aql

The toolkit comes with a builtin measurement subsytem. The individual measurements are represented as frames which may have subframes. Each frame contains information about time elapsed between the frame beginning and the frame end, the time spend in frame excluding subframes, amount of heap memory allocated at the frame begining and at the frame end, the highest ammount of memory allocated while inside the frame, and the highest ammount of memory allocated inside the frame excluding subframes.

Algorithms may be augmented with calls to measurement subsytem marking frame beginings and ends to allow in-depth measurement control. The query language itself comes with a set of commands to start and stop measurement frames as well to organize measurements in more detail.

The query language additionaly provides a command to clear measurements and print the measurement frames as a list or as a tree.

Note: the memory measurements is currently not functional.

Related commands in detail

Measurement frames can fall into following categories ROOT, OVERALL, INIT, FINALIZE, MAIN, AUXILIARY, PREPROCESS, and ALGORITHM. To start a measurement frame one has to decide its category and name.

Syntax: START category MEASUREMENT FRAME name

To stop a measurement frame, no extra information is needed.

Syntax: STOP MEASUREMENT FRAME

Logged measurements can be printed either as a list or as a tree.

Syntax: SHOW MEASUREMENTS AS print type

Logged measurements can be cleared.

Syntax CLEAR MEASUREMENTS

Comments

Aql supports C++-like comments: single-line comment // and (possibly multiline) /* ... */.