Structuring Odin Projects

Odin is a programming language where the code is mainly organized as procedures and packages (a directory of files). However, Odin documentation doesn’t give a lot of advice on how to organize a project, as expected from a general-purpose language. A good project organization enable our systems to be versatile enough to cover all existing and future features, while remaining consistent. It should also allow scaling small subfeatures into larger ones. A good project structure is that it guides us to design our programs in a way to most benefit from the architecture that they run on.

The rule of thumb when structuring applications, is to allow the top-level directories to directly communicate with layer below, but any communication crossing more than one line is frown upon. As an example, project.odin can retrieve packages in lib/tasks. Reaching out from lib/tasks to project.odin is - however - not good.

Try to keep the directory structure as flat as it is possible and manageable. You don’t really need to type more than necessary when opening/saving files - as long as there is sane number of files in a directory. There is no need to multiply subdirectories for the sake of multiplying subdirectories.

The whole idea is to have packages that work as an entry point for your app’s business. Then, all sub-entities for that aspect live there, on inner files and directories. It’s like your lib has sub-libraries. Then your sub-library has a main interface that lives on an entry point package. The main reasoning is thinking about the deletion and moving. If you want to move a package to another directory, all its files should move together. See the entry point purpose working like an index file in a web server directory, and usually, the index files live in the directory as children, not as siblings.

lib/social_network
    └── authentication
            ├── authentication.odin
            ├── google.odin
            ├── facebook.odin
            └── password.odin

Avoid namespacing, use Sufixes

Some kinds of files fit in one or more categories and do not belong exclusively to a common package. For example, errors, your app might have several packages that represent errors. Let’s say you have multiple errors in your app: argument, arithmetic, not found, etc. They are all errors, but they should not belong to a single “Error package”. Instead of creating a namespace, you can use a suffix. Your naming would look like this:

package.ArgumentError
package.ArithmeticError
package.NotFoundError
package.TemporarilyUnavailableError

So they aren’t tied anymore to the unhelpful namespacing and concept “Error”, now you’re free to move them to sub or parent applications as the time needs. As long as your project grows, these files that aren’t together conceptually might pollute your OS directories and make things hard to navigate. So it’s okay to organise your OS files and directories in a way that works better.

Building a Functional Core in Odin

The book Designing Elixir Systems with OTP shows the concept of “Do Fun Things with Big, Loud Worker-Bees” (a mnemonic for understanding the layers). A concept tailored to design libraries with intelligent layers that shape the right data structures, flow from one procedures into the next, and present the right APIs. It shows you how to go beyond simple programming to designing, and that means building the right layers. Embrace data structures that work best and use them to build procedures that perform and compose well, layer by layer, across packages. Test your code at the right place using the right techniques. Layer your code into pieces that are easy to understand. The objective when using these techniques is making the codebases a lot simpler.

Functional Core (Do Fun Things)

This is the “core”, “kernel” or “domain”. Is the main business logic unit that is composed by Data, Functions and Procedures and Tests.

Data

The idea is to rely on inmutable data structures as much as posible, so applications get much easier to maintain and algorithms get much cleaner, achieving harmony between data structures and procedures.

Odin gives developers several data structures. Structs, Maps, Enums, Strings, but also procedures that can be data too. Sometimes using procedures as data params can offer tremendous perfomance wins.

Example

We could store our Square with the data.

Square {
        {5, 0}, {15, 0}
        {15, 0}, {15, 10}
        {15, 10}, {5, 10}
        {5, 10}, {5, 0}
    }

This way works fine, but we could also convert it to a procedure that calculates the values. That way we can avoid passing and coping heavy data in the params.

package Data

import "core:fmt"

Square :: struct {
    x1, y1, x2, y2, x3, y3, x4, y4 : [2]int
}

square :: proc (x: int, y: int, size: int) -> Square {
    return Square{
        {x, y}, {x + size, y}, 
        {x + size, y}, {x + size, y + size},
        {x + size, y + size}, {x, y + size},
        {x, y + size}, {x, y}
    }
}

main :: proc() {
    // We can save the procedure in a variable
    // Or pass the procedure as a param in another procedure
    // This is passing the pointer
    my_procedure_pointer := square

    // Square{x1 = [5, 0], y1 = [15, 0], 
    // x2 = [15, 0], y2 = [15, 10], 
    // x3 = [15, 10], y3 = [5, 10], 
    // x4 = [5, 10], y4 = [5, 0]}
    fmt.printf("%v", my_procedure_pointer(5, 0, 10))
}

We start with a procedure called square. It takes a point (x, y) and a size, and transforms that data to the same square format we saw earlier.

Procedures

Procedures needs certainty and sanitized data. If you were building an application it would be more of a library. The main concept here is CRC, which means: Constructors, Reducers and Converters.

Constructors: Are procedures that creates and sets data structures that will be pased down to Reducers and Converters. This data structure is known as the accumulator or token.
Reducers: These procedures take the accumulator and “reduce it”, applying a pipeline of different operations and procedures until it reaches a state ready for the Converter.
Converter: Takes the accumulator and “convert it” to a final format, that is ready for displaying to the user or pass it to another pipeline of CRC. Example would be a procedure that takes a json data structure and convert it to string, this string will be passed down to the pipeline for saving the contents to a file in disk.

Thanks to this form of organization, we can see our code in a coherent and unified way. By having a common data structure with several functions, we can perform operations and express ourselves with greater readability. Our systems will be easier to understand and maintain in the future.

Consistency is an important quality factor in our systems and using CRC is a great tool to achieve that.

Example

Odin uses packages, procedures and data types, which allows a way to organize our code in what is called CRCs (Constructors, Reducers, and Converters). So we will have procedures to (create) an accumulator, procedures that will perform operations (reducers) with this accumulator and finally procedures that will transform the accumulator into a final format (converters). This is something that has existed for a long time in different programming languages such as Haskell or Lisp. The most important thing you can do in a language is to unite ideas using composition. The idea is to create a pipeline that receives an accumulator as its first parameter and performs a series of reduce operations until reaching the final converter.

package CRC

import "core:fmt"

Person :: struct {
    name : string,
    age : int
}

// Creator
new_person :: proc(name: string, age: int) -> Person {
    person : Person
    person.name = name
    person.age = age
    return person
}

// Reducer
// We use a new structure to avoid mutating the original parameter
rejuvenate_person :: proc(person: Person, years: int) -> Person {
    return new_person(person.name, person.age - years)
}

// Converter, we only show the value we want to.
person_to_string :: proc(person: Person) -> (string, string, int) {
    return "The person %s, now have %d years", person.name, person.age
}

// Pipeline
main :: proc() {
    person := new_person("Camilo", 35)
    person = rejuvenate_person(person, 10)
    out, name, age := person_to_string(person)
    fmt.printfln(out, name, age) // The person Camilo, now have 25 years
}

Example 2

Let’s illustrate this a bit using a treasure map. In this map we are going to give a series of directions north, south, east and west. You can see that we have a creation function that returns a Treasure struct {x, y}, which will be our accumulator data structure. Then we have a series of reducers that modify the struct and return a new struct with the appropriate values. Finally we have our converter that returns a String with a final message.

package CRC

import "core:fmt"

Treasure :: struct {
    x: int,
    y: int
}

// Constructors
new :: proc(x: int, y: int) -> Treasure {
    treasure : Treasure
    treasure.x = x
    treasure.y = y
    return treasure
}

empty :: proc() -> Treasure {
    return new(0, 0)
}

// Reducers
north :: proc(treasure : Treasure) -> Treasure {
    return new(treasure.x, treasure.y - 1)
}

south :: proc(treasure : Treasure) -> Treasure {
    return new(treasure.x, treasure.y + 1)
}

east :: proc(treasure : Treasure) -> Treasure {
    return new(treasure.x - 1, treasure.y)
}

west :: proc(treasure : Treasure) -> Treasure {
    return new(treasure.x + 1, treasure.y)
}

// Converter
show :: proc (treasure : Treasure) -> (string, int, int) {
    return "The treasure is on the coordinates (%d, %d)", treasure.x, treasure.y
}

// Pipeline
main :: proc() {
    treasure := empty() // (0, 0)
    treasure = north(treasure) // (0, -1)
    treasure = north(treasure) // (0, -2)
    treasure = north(treasure) // (0, -3)
    treasure = north(treasure) // (0, -4)
    treasure = west(treasure)  // (1, -4)
    treasure = west(treasure)  // (2, -4)
    treasure = west(treasure)  // (3, -4)
    treasure = south(treasure) // (3, -3)
    treasure = east(treasure)  // (2, -3)
    treasure = east(treasure)  // (1, -3)
    treasure = east(treasure)  // (0, -3)
    out, x, y := show(treasure)
    fmt.printfln(out, x, y) // The treasure is on the coordinates (0, -3)
}

We can compare these techniques with the Object Oriented pattern of Command - Query Segregation. Bertrand Meyer in his book Object Oriented Software Construction.

The fundamental idea is that we should divide an object’s methods into two sharply separated categories:

Queries: Return a result and do not change the observable state of the system (are free of side effects).
Commands: Change the state of a system but do not return a value.

Because the term ‘command’ is widely used in other contexts, is preferred to refer to them as ‘modifiers’, ‘mutators’, or in this case reducers. The really valuable idea in this principle is that it’s extremely handy if you can clearly separate methods that change state from those that don’t. This is because you can use queries in many situations with much more confidence, introducing them anywhere, changing their order. You have to be more careful with modifiers.

Example 3

User @thetarnav points that this can be achieved like the following code too.

Treasure :: [2]int

NORTH :: Treasure{ 0, -1}
SOUTH :: Treasure{ 0, +1}
EAST  :: Treasure{-1,  0}
WEST  :: Treasure{+1,  0}

treasure := \
    NORTH + // (0, -1)
    NORTH + // (0, -2)
    NORTH + // (0, -3)
    NORTH + // (0, -4)
    WEST +  // (1, -4)
    WEST +  // (2, -4)
    WEST +  // (3, -4)
    SOUTH + // (3, -3)
    EAST +  // (2, -3)
    EAST +  // (1, -3)
    EAST    // (0, -3)

Nevertheless is an interesting way of approaching system organization. This is using Array programming techniques where basic operations (+, -, *, etc.) on two fixed-size arrays of the same length are done element-wise. One of the goodies of Odin.

Tests

With proper structure of your projects testing will be much simpler. It will be possible to represent testing concepts in any way you need.

Final Thoughts

The internal building blocks of any project can be dividen in datatypes, procedures and tests. The datatypes guide the structure of components and interactions between procedures. Those procedures are divided along the obvious lines of purpose (CRC). And by separating the project in layers we can use tests to verify what we’ve done, using conventional techniques to test the core and other layers.

Draupnir Templates