Modules

This concept guide introduces the basic syntax for modules in Rel through simple and more advanced examples.

Introduction

Modules in Rel provide a way to organize and structure your data modeling, offering some distinct advantages:

• Related definitions can be grouped together.
• Models and database logic can be parameterized and reused.
• Names can be local to each module, which prevents clashes and keeps the top-level namespace clean.
• Relations can be renamed before they are used.

Simple Example

The basic module syntax groups relations under a common namespace. For example:

// model
 
module store
    def office = {"Boston"; "New York"; "Los Angeles"; "Chicago"}
    def product = {(1, "Laptops"); (2, "Desktops"); (3, "Phones")}
end

A module is a relation that can be thought of as containing nested relations.

// read query
 
store

In the example above, office and product are nested inside store, and available as store[:office] and store[:product]:

// read query
 
store[:office]

// read query
 
store[:product][1]

Instead of writing store[:office], you can write store:office. This is called a qualified name, and is treated as if it were an indivisible identifier. The colon in store:office is not an operator — it is an intrinsic part of the Symbol :office, which is concatenated with store.

Similarly, instead of store[:product][1], you can use a qualified name and write store:product[1]. The shorter versions are preferred:

// read query
 
def R = equal(store[:office], store:office)
def output = R
 

The syntax for module declarations is just a convenient notation. The example above could also be written as:

def store:office = {"Boston"; "New York"; "Los Angeles"; "Chicago"}
def store:product = {(1, "Laptops"); (2, "Desktops"); (3, "Phones")}

You could write it even more directly:

def store = :office, {"Boston"; "New York"; "Los Angeles"; "Chicago"}
def store = :product, {(1, "Laptops"); (2, "Desktops"); (3, "Phones")}

In the interest of readability, you should use the module syntax, rather than any alternatives.

Importing Module Names: with

You can choose to import some of the definitions of a module into other namespaces and optionally rename them, using statements of the form with <module> use <relation> [as <newname>]. For example:

// read query
 
with store use office
def output = count[office]

This example shows how you can rename relations when you import:

// read query
 
with store use product as p
def output = p[1]

Using Modules Inside Other Modules

You can use existing module definitions when defining new modules. For example:

// read query
 
module summary
    with store use office
    def office_count = count[office]
    def product = store:product
end
 
def output = summary

Note that while store:office was used to define office_count, it did not become part of summary. By contrast, store:product is part of summary, because of the definition def product = store:product.

Variable Scope

When variable names clash, the innermost scope takes precedence. For example:

// read query
 
def a = 10
 
module m
    def a = 20
    def b = a + 5
end
 
def output = m

Imports can be shadowed by a new definition that follows them, which will take precedence. In this case, a warning is printed:

// read query
 
module m1
     def a = 1
end
 
with m1 use a
 
def a = 5
 
def output = a

The warning states: “‘a’ is imported but not used because there is a declaration of the same name in the same scope.”

If the def a = ... comes before the with, the import takes precedence in the code that follows it.

// read query
 
module m1
     def a = 1
end
def a = 5
 
with m1 use a
 
def output = a

Nested Modules

Modules can be nested. In other words, modules can be defined within modules. This is especially useful when data are fully normalized into Graph Normal Form:

// read query
 
module person
    def ssn = 123-45-6789
    module name
        def first = "John"
        def middle = "Q"
        def last = "Public"
    end
    module birth
        def city = "Pittsburg"
        def state = "PA"
        def country = "USA"
        def date = parse_date["2000-01-01", "Y-m-d"]
    end
end
 
def output = person:birth:city

Multiple Definitions

Like all relations in Rel, modules can be made up of a number of separate definitions:

// read query
 
module m2
    def a = 1
end
 
module m2
    def b = 1
end
 
def output = m2

With this feature, you can, for example:

• Define large or complex features of a given module separately, even in different installed sources.
• Extend an existing module for the purposes of a particular query, without persisting the change.

You can also take the union of two separate modules. For example:

// read query
 
module A
    def a = 1
end
 
module B
    def a = 1
    def b = 1
end
 
def AB = A ; B
 
def output = AB

This lets you extend a given module with functionality defined in another.

Parameterized Modules

Modules can be made more reusable by parameterizing them. For example, you can build a simple module that collects a few statistics for a relation R:

// model
 
@outline
module my_stats[R]
    def my_minmax = (min[R], max[R])
    def my_mean = mean[R]
    def my_median = median[R]
end

// read query
 
def output = my_stats[{1; 2; 3; 5; 8; 13; 100}]

The module parameter R can be instantiated with any relation that has a numeric range. For example:

// read query
 
def pop = {("a", 10); ("b", 100); ("c", 20); ("d", 23)}
def output = my_stats[pop]

In a parameterized module each parameter represents a relation. The relation may be a singleton, in which case the argument used in instantiating the module may be a single data value or an individual variable. If a parameter is intended to represent a relation that is not a singleton, then its name must begin with an upper-case letter.

The definition of a parameterized module whose parameters are not singletons should be preceded by one of two annotations: @outline or @inline. The recommended annotation is @outline. Unlike @inline, it allows the module to contain recursive definitions. The recursive definitions need not be annotated with additional occurrences of @outline.

The following example shows a module that is intended to be parameterized with a singleton relation, i.e., a single data value or an individual variable. The name of the parameter may begin with a lower-case letter, and an annotation is not necessary:

// read query
 
module mymodule[x]
    def multiply[y] = y * x
    def divide[y] = y / x
end
 
def m = mymodule[10]
 
with mymodule[20] use divide as divides_twenty
def output:first = m:divide[240]
def output:second = divides_twenty[120]
 

The following is a simple example of a module that features a recursive relation, transitive. Notice that there is no need to annotate the definition of transitive with @outline:

// read query
 
// Define the common extensions of a binary relation.
@outline
module extensions[R]
  def id = R
  def symmetric = R ; transpose[R]
 
  def reflexive = x, y : (first[R]; second[R])(x) and x = y
  def reflexive = R
 
  def transitive = R
  def transitive = R . transitive
 
  def closure = reflexive ; transitive
end
 
def extended = extensions[{("a", "A"); ("b", "B"); ("B", "b")}]
 
def output:id = extended:id
def output:trans = extended:transitive

Sometimes it may make sense to define modules whose parameters are constants. Here is an example, but it is not a recommendation for structuring data this way:

// read query
 
module country["Ireland"]
  def capital = "Dublin"
  def access_to_sea = "Atlantic"
end
 
module country["France"]
  def capital = "Paris"
  def access_to_sea = "Atlantic"; "Mediterranean"
end
 
def output:France = country["France"]
def output:Ireland:capital = country["Ireland"][:capital]

Multiple Parameters

Modules can have any number of parameters representing singleton or general relations. A module definition with four parameters can be written as:

module M[A, B, x, y]
    // ....
end

The higher-order parameters A and B are for general relations. The first-order parameters x and y are for singleton relations.

In the example below, csv_column_filter[Data, Column] is defined as a parameterized module that acts as a column filter operation over the CSV data Data.

// read query
 
// Define a module that filters columns.
@outline
module csv_column_filter[Data, Column]
    def result = Column <: Data
end
 
// Define the toy data in a string.
def config:data = """
name,country,city
Liam,USA,"Portland, ME"
Olivia,Canada,Toronto
Noah,Mexico,Yucatan
Emma,USA,"Washington, D.C."
"""
 
// Load the string as CSV data.
def my_csv = load_csv[config]
 
// Filter columns and display the result as a CSV table.
def output = ::std::display::table[
    csv_column_filter[my_csv, {:name; :city}][:result]
]

The module csv_column_filter is invoked for the CSV data my_csv, which has been created with load_csv, and is filtered on the columns “name” and “city” with the prefix join operator, <:. This creates a relation with tuples such as (:result, :city, 2, "Portland, ME"). The first element — that is, :result — is dropped, and the remaining information is rearranged with the library relation table.

See the declaration of BipartiteGraph in section Example: Graph Modules for an example of parameterized modules in the context of graphs.

Defining Templates

Thanks to the fact that modules can have multiple parameters, they can also be used to define templates for tasks such as transformations. This means an entire class of transformations can be defined with a parameterized module. Custom names can be given to specific transformations within this class, allowing users to build a complex transformation in a highly modularized way.

In the example below, the module csv_filter acts as a template for filtering columns and rows of data in GNF.

// read query
 
// Define a template for filtering data in the GNF format.
@outline
module csv_filter[Data, Column, n_row]
    def top(c, r, v) = Data(c, r, v) and Column(c) and r <= n_row
    def bottom(c, r, v) = Data(c, r, v) and Column(c) and r > n_row
end
 
// Define a custom filter with concrete filtering parameters.
@outline
def my_filter[Data] = csv_filter[Data, {:name; :city}, 2]
 
// Define the toy data.
module my_csv
    def name {(1, "Liam"); (2, "Olivia"); (3, "Noah"); (4, "Emma")}
    def country {(1, "USA"); (2, "Canada"); (3, "Mexico"); (4, "USA")}
    def city {(1, "Portland, ME"); (2, "Toronto"); (3, "Yucatan"); (4, "Washington, D.C.")}
end
 
// Apply the custom filter.
def my_csv_filtered = my_filter[my_csv][:top]
 
// Display the result.
def output = ::std::display::table[my_csv_filtered]

The relation top within csv_filter selects the first n_row rows in the CSV data. The accompanying relation bottom discards the first n_row rows. They are designed — in contrast to top and bottom in the Standard Library — to filter data in GNF format.

The relation my_filter is defined to be a specific custom filter based on the csv_filter template. It selects or drops two rows, and only from the columns “name” and “city”.

Once my_filter is defined — within the query or installed in the database — it can be applied to any relation, like my_filter[my_csv], without the need to specify the filter parameters again. The argument :top effectively specified that the relation top within csv_filter should be used. Without it, both the filters (top and bottom) are applied to my_csv.

Example: Graph Modules

Since modules are relations themselves, they can be used to parameterize other modules.

Consider a module that constructs a complete graph for a set of nodes N; that is, a graph where each node is connected to all other nodes:

// model
 
@outline
module CompleteGraph[N]
    def node = N
    def edge = N,N
end

Two graphs are defined in the example below. The first is a bipartite graph, with a set of nodes, M, each of which is connected to all nodes from the set N. The second is a cycle graph, with a set of nodes, N. Each node in N is connected to one “next” node, forming a loop:

// model
 
@outline
module BipartiteGraph[M, N]
    def node = M; N
    def edge = M, N
end
 
@outline
module CycleGraph[N]
    def node = N
    def edge(a in N, b in N) =
            sort[N](x, a)
        and sort[N](y, b)
        and y = x % count[N] + 1
        from x, y
end

The module GraphProperties defined below computes some basic properties of a graph module G:

// model
 
@outline
module GraphProperties[G]
    def outdegree[v in G:node] = count[v1 : G:edge(v, v1)] <++ 0
    def indegree[v in G:node] = count[v1 : G:edge(v1, v)] <++ 0
    def edge_count = count[G:edge] <++ 0
end

The following code instantiates the BipartiteGraph module and passes it as a parameter to the GraphProperties module to compute its properties:

// read query
 
def graph = BipartiteGraph[{"l1"; "l2"}, {"r1"; "r2"; "r3"}]
 
def output = GraphProperties[graph]
 
 

Here are more examples showing how you can use these definitions:

// read query
 
def cg = CompleteGraph[range[1, 5, 1]]
def cg_props = GraphProperties[cg]
 
def bg = BipartiteGraph[{1; 2}, {3; 4; 5}]
def bg_props = GraphProperties[bg]
 
def cycleg = CycleGraph[{"a"; "b"; "c"; "d" ; "e"}]
def cycleg_props = GraphProperties[cycleg]
 
module output
    def complete_edge_count = cg_props:edge_count
    def bipartite_edge_count = bg_props:edge_count
    def cycle_edge_count = cycleg_props:edge_count
end
 

Types Defined in Modules

A module can contain definitions of entity types and value types.

Here is a very simple example:

// read query
 
module m
  entity type Country = String
end
 
def scandinavia(x, c) {
    c = m:^Country[x] and {"Sweden"; "Denmark"; "Norway"}(x)
}
def output = scandinavia

See Value Types in Modules for more information.

Integrity Constraints in Modules

Currently, basic integrity constraints are supported in non-parameterized modules. They are also supported in modules that are parameterized only with constants. Here’s an example:

// read query
 
module mymodule
    def R = {1; 2}
    ic {count[R] = 2}
end
 
def output = mymodule

For more details, see Integrity Constraints in Modules in the Integrity Constraints concept guide.

Imported CSV Files and JSON Documents Are Modules

The result of importing CSV and JSON data can be viewed as a module.

CSV Data as Modules

Loading a CSV file into a relation csv_data gives a module of the form:

module csv_data
   def column_one[pos] = ...
   def column_two[pos] = ...
   ...
end

The relation csv_data:column_one maps positions to values for the first column in the CSV file, the same for csv_data:column_two and the second column and so on. See the CSV Import Guide for further details. For example:

// read query
 
def config:data = """
order_id,customer,price
1,92819,3.4
3,78271,1.6
"""
 
def csv_data = load_csv[config]
def output = csv_data

This relation is equivalent to the module:

module order_csv[pos]
    def order_id = ...
    def customer = ...
    def total_price = ...
end

This in turn is equivalent to:

def order_csv[pos, :order_id] = ...
def order_csv[pos, :customer] = ...
def order_csv[pos, :total_price] = ...

JSON Data as Modules

For JSON data, the input JSON structure is naturally reflected in the resulting Rel module structure, with nested JSON structures becoming nested modules, recursively. JSON names become Rel meta values, and values become scalars (single values) or nested modules.

For example:

// read query
 
def config:data = """
{
  "name": "John",
  "address": {"state": "WA", "city": "Seattle"},
   "age": 21
}"""
 
def person = load_json[config]
def output = person

Summary

In summary, modules in Rel allow you to organize and model your data, and they also provide certain distinct advantages. Some of these advantages include grouping relations with common definitions or using existing definitions to define new modules. Moreover, modules can be parameterized and used to construct complete graphs. Lastly, imported CSV and JSON data can also be viewed as Rel modules.