Building a Cross Platform Project

A project with a portable codebase means that it can be built to target different platforms and offer its functionality on various systems.

When code is referred to as “portable”, it can imply different things. For example:

• It may not explicitly use platform-specific functions or libraries
• It may abstract away platform specificities and use these abstractions in most of its codebase
• It may be implemented in a high-level language that intrinsically provides platform abstractions

Regardless of the strategy employed, some abstraction exist over platform-specific services, concepts and data structures.

In any case, when building the project’s codebase, one is transforming the code into a format that will be allowed to run on the target machine.

This article is the second in a series of posts that will dive into the development environment of a cross-platform project.

This time, we will look into the process of building a cross-platform project by identifying the different possibilities for building and how this can be organized into a full solution to make this process as painless as possible.

Building a Build Model for C/C++

When dealing with compiled languages like C/C++, two critical components collaborate during the build process: the compiler and the linker.

We may think of them as tools that crunch code and assemble libraries into some format that machines can easily understand.

The critical part of a cross-platform project is that format: the machine on which our compiled code will run needs to understand it.

Moreover, any pieces the project does not provide and depends on such as libraries need to be made available in the machine-understood format so it can be found at linking, or for runtime libraries at execution.

Simple Model: Host = Target

This sounds rather trivial, right? All of this is accessible when building on the same platform as you’re targeting given you’re using the toolchain that was made for your platform. Let’s build a simple model of the de-facto toolchains for the three main desktop Operating Systems (OS): Windows, macOS and GNU/Linux:

Host		Target
	Windows	Mac	GNU/Linux
Windows	MSVC
macOS		Apple-clang
GNU/Linux			gcc

The point here is that we are in the simplest scenario: use the main toolchain for the platform and you’re good!

Is Simpler Better?

What we gain in simplicity, we lose in flexibility. The one obvious limitation of this model is that it forces you to have access to a machine instance of each target platform.

Virtual Machines - Rather than dealing with different physical machines, you may use Virtual Machines (VM) running the target environment on a single physical machine. All that is needed is to provide access to the codebase directory residing on the host machine, for example by sharing the folder over network with appropriate permissions, and issue a build command in the VM, either directly or indirectly through SSH or some build manager.

One major advantage is the fact that your application and its test suite can be run in those virtualized environments.

However, some platforms are harder to virtualize than others, e.g., macOS, but it’s doable. Be aware though that with this method, you’ll incur some overhead when accessing the files in the virtualized environment and be limited to the VM allocated resources when building (CPU cores and RAM). These are also resources that your host system cannot touch to do its own work.

Containers - One may be tempted to opt for containers, which are more lightweight given they do not virtualize the kernel, instead relying on the host’s. The nature of containers means that running a container of a different platform from the host’s is tricky: the requirements for running an executable that is of a different type and relies on OS services would need to be fulfilled by some implementation built on top of the host system. While this can be done (Wine, WSL 1), container technologies typically opt to rely on a VM instead (Docker, WSL 2 using Hyper-V).

This brings us back to VMs, with container management as additional overhead.

Cross Compilation Model: Host != Target

To fill in the blanks of the simple model, we’ll need a new type of toolchain: one that targets another platform than the one it runs on. This is called a cross compiler.

Note that the build model that we’re elaborating does not take into account different architectures. This is left to the cross-native toolchains to provide a specific cross compiler as part of their suite: one that allows to build for the same OS, but targets a different architecture, e.g., building for arm64 on a amd64 host.

Host		Target
	Windows	Mac	GNU/Linux
Windows	MSVC	OSXCross through WSL	gcc through WSL
macOS	gcc through MinGW-w64	Apple-clang	gcc cross-compiler*
GNU/Linux	gcc through MinGW-w64	OSXCross	gcc

*I was not able to find a prebuilt GCC cross-compiler for macOS, but a dedicated developer can build it from source.

Whew! We have a full build model that includes cross compilers, so now we can build anything from anywhere, right? Well, not exactly.

The first major drawback has to do with licensing, especially when it comes to Apple’s SDK, which is required for building macOS applications. Not only you’re not technically allowed to install such SDK on a non-Apple device, you’ll need an Apple machine to sign the application so that it is not flagged as untrusted.

Similar issues arise when using any platform SDK that has its headers protected behind some strict licensing. MinGW-w64 works around this by using the documentation available from MSDN to maintain a standalone set of headers for the Win32 API.

Secondly, you will need to make sure cross compilers are correctly supported by whatever build system you are using. Following on the first post in the series, if you are using Meson this is supported and there is a good list of cross-files offered in the repository.

Lastly, there’s the problem of running the compiled binary: you will need to use an adapter layer discussed previously (Wine, WSL), or resort to a VM. Naturally, if the architecture is different, then there’s just no way to avoid emulation.

Why Would Anybody Do This?

A single advantage may be sufficient for your situation: speed. Because the cross compiler is running natively, it has access to your full system’s power. Parallel jobs run across the full array of cores and RAM can be filled with any compiler or linker process.

Cross compilation is a good tool when iterating quickly, especially when your main driver is checking that your code builds correctly on all platforms. A centralized configuration may also help with maintainability and uniformity, whereas dealing with VMs needs tweaking in different environments.

You’re also not forced to configure everything by hand, tools such as crosstool-NG aim to streamline the process of generating a cross toolchain. It really depends on the array of platforms you are targeting: some make it hard, some make it easier (e.g., Raspberry Pi).

In the end however, you’ll have to ask yourself: is it worth the hassle?

A Different Way to Build

There is one target I have not mentioned as it relies on a completely different model. Instead of compiling your code to an executable binary format that is to be executed directly by the system, another solution is to use a virtualized execution environment. This environment implements a virtual machine on top of the host system, and it completely abstracts away the host in favor of a unified machine model on which the code relies.

This is not new: it’s the approach taken by the Java Virtual Machine, and has recently regained traction with the development of WebAssembly. Some interpreted languages use a similar strategy by turning the code into some bytecode that can be directly interpreted or Just-In-Time (JIT) Compiled into lower-level instructions for the VM, with some optimizations thrown in.

When it comes to C/C++, WebAssembly promises some of the advantages that made the JVM a success in the Write-Once-Run-Everywhere scheme. Unfortunately, a lot of critical features are missing, leading to some parts missing to implement a complex application. However, the promise is there: once a unified API has been accepted and is implemented into the VM running the code: WebAssembly may be a good option to support different platforms without even having to compile for each of them.

Reliance on external runtime platforms, as it is for browsers and web applications today, offers its own can of worms though and surely will have its own set of cross platform challenges with platform-specific extensions and partial support of standards.