Julia Metaprogramming

Julia represents its code as data structures within the language, allowing us to write programs that manipulate other programs.

Metaprogramming can also be simply understood as writing code that generates other code.

>

Metaprogramming (English: Metaprogramming) refers to the writing of computer programs that can treat other programs (or themselves) as their data or perform part of the work at compile time that would normally be done at runtime. In most cases, this allows programmers to achieve higher productivity or provide greater flexibility to handle new situations without recompiling.

The language used to write metaprograms is called a metalinguistic abstraction. The language of the manipulated program is called the "target language." The ability of a programming language to be its own metalinguistic abstraction is called "reflection" or "self-reference."

-- Wikipedia

Julia Source Code Execution Stages

1. Parse Raw Julia Code: The Julia parser first parses the string to obtain an Abstract Syntax Tree (AST), which is a structure that contains all the code in an easily manipulable format.

2. Execute Parsed Julia Code: At this stage, the parsed Julia code is executed.

When we type code into an interactive programming environment (REPL) and press Enter, these two stages are executed.

Using metaprogramming tools, we can access the Julia code between these two stages, i.e., after the source code is parsed but before it is executed.

Program Representation

Julia provides a Meta class where you can parse a string with Meta.parse(str) and return Expr with typeof(e1):

Example

julia> prog = "1 + 1"
"1 + 1"
julia> ex1 = Meta.parse(prog)
:(1 + 1)

julia> typeof(ex1)
Expr

Returning :(1 + 1), this return value consists of a colon followed by the expression, and typeof(ex1) returns Expr.

An Expr object contains two parts (ex1 has head and args attributes):

A symbol object that identifies the type of the expression.

Example

julia> ex1.head
:call

The other part is the arguments of the expression, which can be symbols, other expressions, or literals:

Example

julia> ex1.args
3-element Vector{Any}:
 :+
 1
 1

Expressions can also be constructed directly with Expr:

Example

julia> ex2 = Expr(:call, :+, 1, 1)
:(1 + 1)

The two expressions constructed above: one through parsing and one through direct construction, are equivalent:

Example

julia> ex1 == ex2
true

Expr objects can also be nested:

Example

julia> ex3 = Meta.parse("(4 + 4) / 2")
:((4 + 4) / 2)

We can also use Meta.show_sexpr to view the expression, the following example shows a nested Expr:

Example

julia> Meta.show_sexpr(ex3)
(:call, :/, (:call, :+, 4, 4), 2)

Symbols

We can store an unevaluated but parsed expression by prefixing it with a colon :.

Example

julia> ABC = 100
100

julia> :ABC
:ABC

Quoting the entire expression:

Example

julia> :(100-50)
:(100 - 50)

Quoting arithmetic expressions:

Example

julia> ex = :(a+b*c+1)
:(a + b * c + 1)

julia> typeof(ex)
Expr

Note that equivalent expressions can also be constructed using Meta.parse or directly with Expr:

Example

julia> :(a + b*c + 1) ==
       Meta.parse("a + b*c + 1") ==
       Expr(:call, :+, :a, Expr(:call, :*, :b, :c), 1)
true

Multiple expressions can also be quoted within a quote ... end block.

Example

julia> ex = quote
               x = 1
               y = 2
               x + y
           end
quote
    #= none:2 =#
    x = 1
    #= none:3 =#
    y = 2
    #= none:4 =#
    x + y
end

julia> typeof(ex)
Expr

Executing Expressions

After parsing the expression, we can use the eval() function to execute it:

Example

julia> ex1 = :(1 + 2)
:(1 + 2)

julia> eval(ex1)
3

julia> ex = :(a + b)

julia> eval(ex)
ERROR: UndefVarError: b not defined
[...]

julia> a = 1; b = 2;

julia> eval(ex)
3

Abstract Syntax Tree (AST)

The Abstract Syntax Tree (AST) is a structure that represents the abstract syntactic structure of source code.

It represents the syntactic structure of the program's source code in a tree-like format, where each node in the tree represents a construct in the source code.

We can view the hierarchical structure of an expression with the dump() function:

Example

julia> dump(:(1 * cos(pi/2)))
Expr
   head: Symbol call
   args: Array{Any}((3,))
      1: Symbol *
      2: Int64 1
      3: Expr
         head: Symbol call
         args: Array{Any}((2,))
            1: Symbol cos
            2: Expr
               head: Symbol call
               args: Array{Any}((3,))
                 1: Symbol /
                 2: Symbol pi
                 3: Int64 2

Interpolation

Constructing Expr objects directly with arguments can be powerful, but it can be tedious compared to Julia syntax. As an alternative, Julia allows literal or expression interpolation into quoted expressions. Expression interpolation is indicated by a prefix $.

In this example, the value of variable a is interpolated:

Example

julia> a = 1;

julia> ex = :($a + b)
:(1 + b)

Interpolation into non-quoted expressions is not supported and results in a compile-time error:

julia> $a + b
ERROR: syntax: "$" expression outside quote

In this example, the tuple (1,2,3) is interpolated into a conditional test:

julia> ex = :(a in $:((1,2,3)) )
:(a in (1, 2, 3))

Using $ for expression interpolation is intended to resemble string interpolation and command interpolation. Expression interpolation makes it convenient and readable to construct complex Julia expressions programmatically.

Macros

Macros provide a mechanism to include generated code in the final body of a program. Macros map a tuple of arguments to a returned expression, and the resulting expression is compiled directly without requiring runtime eval calls. Macro arguments can include expressions, literals, and symbols.

Here is a very simple macro:

Example

julia> macro sayhello()
           return :( println("Hello, world!") )
       end
@sayhello (macro with 1 method)

Macros in Julia have a dedicated character @ (at-sign), followed by the unique macro name declared using the macro NAME ... end form. In this example, the compiler replaces all occurrences of @sayhello with:

:( println("Hello, world!") )

When @sayhello is entered in the REPL, the interpreter executes it immediately, so we only see the computed result:

julia> @sayhello()
Hello, world!

Now, consider a slightly more complex macro:

Example

julia> macro sayhello(name)
    return :( println("Hello, ", $name) )
end
@sayhello (macro with 1 method)

This macro takes one argument name. When @sayhello is encountered, the quoted expression is expanded and the value from the argument is inserted into the final expression:

Example

julia> @sayhello("human")
Hello, human

We can use the macroexpand function to see the quoted return expression:

Example

julia> ex = macroexpand(Main, :(@sayhello("human")) )
:(Main.println("Hello, ", "human"))

julia> typeof(ex)
Expr

We can see that the literal "human" has been inserted into the expression.

There is also a macro @macroexpand which might be more convenient than the macroexpand function:

Example

julia> @macroexpand @sayhello "human"

:(println("Hello, ", "human"))