Bitwise Operation Library and Unit Test

In this chapter, we'll implement a bitwise operation library. In Ymir, where we interact directly with hardware, bitwise operations are frequently required. Zig doesn't provide convenient syntax or standard library for bitwise operations, so we'll implement our own utilities. We'll also introduce how to write unit tests in Zig.

Table of Contents

• Bitwise Operation Library
  • Setting a Bit
  • Testing a Bit
  • Concatenating Two Integers
• Creating Unit Test
  • Build Configuration
  • Creating a Test Case
• Summary

Bitwise Operation Library

After creating the bits.zig file, make sure to mark it as pub in ymir.zig:

pub const bits = @import("bits.zig");

This allows you to call the functions implemented in bits.zig from any file like this:

const ymir = @import("ymir");
const bits = ymir.bits;

bits.foobar();

Setting a Bit

The first function we'll implement is tobit(), which takes an integer N and returns a value with only the N-th bit set. This is useful, for example, when creating a mask like 0b0000_1000 to disable IRQ 3. While the intuitive implementation would be 1 << N, things aren't quite that simple in Zig. You need to explicitly specify the type of 1, ensure that N is within the allowed shift range, and call @intFromEnum() when N is an enum. The purpose of tobit() is to hide these tedious details behind a simple interface.

tobit() is designed to be callable with any integer type T.

pub fn tobit(T: type, nth: anytype) T {
    const val = switch (@typeInfo(@TypeOf(nth))) {
        .Int, .ComptimeInt => nth,
        .Enum => @intFromEnum(nth),
        else => @compileError("tobit: invalid type"),
    };
    return @as(T, 1) << @intCast(val);
}

The initial switch distinguishes behavior based on the type T. Currently, it supports integers and enums. If the value is an enum, it uses @intFromEnum() to convert it to an integer. Finally, it shifts 1 to the left by the resulting integer value.

You can use this function like this:

const Irq = enum(u8) { keyboard = 1 };
const irq: Irq = .keyboard;
const mask = bits.tobit(u8, irq);

Testing a Bit

Next, we’ll implement the isset() function, which tests if a specific bit is set in a given integer value:

pub inline fn isset(val: anytype, nth: anytype) bool {
    const int_nth = switch (@typeInfo(@TypeOf(nth))) {
        .Int, .ComptimeInt => nth,
        .Enum => @intFromEnum(nth),
        else => @compileError("isset: invalid type"),
    };
    return ((val >> @intCast(int_nth)) & 1) != 0;
}

This function is almost the same as tobit(). However, unlike left shift, right shift works regardless of the integer type of val, so there's no need to take T as an argument.

Concatenating Two Integers

It is often necessary to take two u32 integers a and b and concatenate them into a single u64 integer. For example, the WRMSR instruction writes a value to the MSR by concatenating EDX and EAX. Normally, you would write something like @as(u64, a) << 32 | @as(u64, b), but this code can be a bit cumbersome. The concat() function abstracts this operation away:

pub inline fn concat(T: type, a: anytype, b: @TypeOf(a)) T {
    const U = @TypeOf(a);
    const width_T = @typeInfo(T).Int.bits;
    const width_U = switch (@typeInfo(U)) {
        .Int => |t| t.bits,
        .ComptimeInt => width_T / 2,
        else => @compileError("concat: invalid type"),
    };
    if (width_T != width_U * 2) @compileError("concat: invalid type");
    return (@as(T, a) << width_U) | @as(T, b);
}

This function is slightly more complex than the previous ones. By specifying the type of b as @TypeOf(a), it enforces that both arguments a and b have the same type. Since anytype parameters are implicitly comptime, you can specify the type of one argument using @TypeOf() of another argument. Also, the width of the resulting type T (width_T) must be exactly twice the width of the type of a and b (width_U). If this condition is not met, a compile-time error is generated using @compileError(). Note that this function does not support enums.

You can use like this:

const a: u32 = 0x1234_5678;
const b: u32 = 0x9ABC_DEF0;
const c = bits.concat(u64, a, b); // 0x1234_5678_9ABC_DEF0

Creating Unit Test

When you create a library like this, it's only natural to want to write tests; that's just human nature.

Writing tests for Surtr or Ymir is not a trivial task. This is because Surtr and Ymir run on bare metal, so they cannot be tested in userland. The only practical approach to testing is to perform assertions during runtime to test certain conditions. In this series, we do not cover testing executable files that include architecture-dependent code. If you are interested, I encourage you to try implementing such tests on your own.

pub const is_test = @import("builtin").is_test;

On the other hand, writing tests for libraries like the one implemented here is straightforward. Since they do not depend on architecture-specific code, they can be run in userland. Below, we write unit tests for bits.zig.

Build Configuration

First, add a build target for unit tests:

const ymir_tests = b.addTest(.{
    .name = "Unit Test",
    .root_source_file = b.path("ymir/ymir.zig"),
    .target = b.standardTargetOptions(.{}),
    .optimize = optimize,
    .link_libc = true,
});
ymir_tests.root_module.addImport("ymir", &ymir_tests.root_module);

The root file for tests is set to ymir/ymir.zig. Since the tests should run on the host OS userland, unlike the Ymir executable, we specify the default b.standardTargetOptions(.{}) as .target. Additionally, we declare Ymir as a dependency.

This only adds the target for the unit test, but there is no target to actually run the tests yet. Let's add a target for running them as well:

const run_ymir_tests = b.addRunArtifact(ymir_tests);
const test_step = b.step("test", "Run unit tests");
test_step.dependOn(&run_ymir_tests.step);

This allows you to run the unit tests by specifying the test target.

Creating a Test Case

In Zig, tests are written inside test {} blocks. As an example, let's write a test for tobit() function:

const testing = @import("std").testing;

test "tobit" {
    try testing.expectEqual(0b0000_0001, tobit(u8, 0));
    try testing.expectEqual(0b0001_0000, tobit(u8, 4));
    try testing.expectEqual(0b1000_0000, tobit(u8, 7));
}

Let's run the test:

> zig build test --summary all
Build Summary: 3/3 steps succeeded
test success
└─ run Unit Test success 678us MaxRSS:1M
   └─ zig test Unit Test Debug native success 1s MaxRSS:204M

This is quite confusing, but actually, no tests have been executed... For now, Zig doesn't provide a straightforward way to list the tests that have run. Because of this, it's not immediately obvious whether tests have been run or not.

Putting that aside, the reason tests are not running is that bits.zig itself is not being evaluated. In Zig, there's a principle that "nothing is evaluated until it is referenced." In this case, the root file of the unit test, ymir.zig, does not reference bits.zig. Although @import("bits.zig") is present, its contents are not actually used, so Zig does not evaluate the file. As proof, if you add clearly invalid code like hogehoge at the end of bits.zig and run zig build test, no error occurs. Because it’s not referenced, it’s not evaluated, and until evaluated, even invalid code causes no issues.

This principle is basically useful. Unnecessary code is excluded from the final binary, and since it’s not evaluated, compile takes less time. However, this principle can be a hindrance when it comes to testing. Sometimes you want to run tests on implemented functions that are not yet used anywhere else.

To address this, Zig provides a function called testing.refAllDecls(). This function evaluates all fields defined in the specified type (in Zig, files are treated like types). Evaluating means that any tests within will also be executed. Add the following to the root file:

const testing = @import("std").testing;

test {
    testing.refAllDeclsRecursive(@This());
}

By specifying refAllDeclsRecursive(), not only are all fields defined by @This() evaluated, but also the fields referenced by those fields are recursively evaluated. This ensures that tests in bits.zig will also be executed:

> zig build test --summary all
Build Summary: 3/3 steps succeeded; 4/4 tests passed
test success
└─ run Unit Test 2 passed 1ms MaxRSS:1M
   └─ zig test Unit Test Debug native success 1s MaxRSS:206M

This time, you will see the message 2 passed. The tests are now properly executed.

Summary

In this chapter, we implemented a bitwise manipulation library and covered how to write unit tests for it. Although we only wrote tests for tobit(), I encourage you to add unit tests for the other two functions as well. Moving forward in this series, we will omit unit test implementations, but when developing for real, it’s a good idea to leverage Zig’s testing capabilities. In the upcoming chapters, we will use the library we implemented here throughout the series.