Platform Gotchas & Internals¶
Systems programming across 10 operating system families requires navigating subtle architectural differences, historical ABI decisions, and kernel quirks.
This document serves as an engineering post-mortem of how cais resolves these challenges.
1. Winsock vs POSIX: The fd_set Divergence¶
One of the most dangerous portability bugs in C networking involves select() and fd_set.
In POSIX (Linux, BSD, macOS)¶
On POSIX systems, fd_set is a fixed-size bitmask containing an array of integers (typically totaling 1024 bits).
* Bit n corresponds to file descriptor integer n.
* If a high-volume server process has opened many files (pipes, logs, database connections) and receives socket file descriptor 1024 or higher, calling FD_SET(1024, &fds) writes past the bounds of the stack bitmask.
* The Result: Silent stack corruption and random memory crashes.
* The cais Solution: On POSIX, cais uses native poll(2) (cais_posix_poll), which accepts a dynamic array of struct pollfd. High descriptor IDs (fd >= 1024) are handled transparently with zero risk of buffer overflows.
In Winsock (Windows NT 3.51 to 11)¶
Microsoft chose a completely different ABI for Winsock in 1993:
typedef struct fd_set {
u_int fd_count; /* Number of sockets currently in array */
SOCKET fd_array[FD_SETSIZE]; /* Explicit array of SOCKET handles */
} fd_set;
- In Winsock,
SOCKETvalues are arbitrary 32/64-bit kernel object pointers, not sequential indices. FD_SETdoes not index a bitmask; it simply appends the handle tofd_arrayand incrementsfd_count.- Redefining
FD_SETSIZE: Because it is an explicit array, redefining#define FD_SETSIZE 1024before including<winsock2.h>safely expands the array capacity from the default 64 to 1024 sockets without any ABI breakage or stack corruption.
2. Windows NT 3.51 PE Loader & Dynamic WSAPoll¶
When Microsoft introduced WSAPoll in Windows Vista (NT 6.0), developers attempted to use it to replace select().
However, compiling code that calls WSAPoll() statically bakes an import reference into the PE binary's .idata section:
If that executable is launched on Windows NT 3.51, Windows NT 4.0, Windows 2000, or Windows XP, the Windows PE Loader inspects the import table, discovers that WSAPoll does not exist in WS2_32.DLL, and immediately aborts execution with a fatal dialog before main() is ever reached:
"The procedure entry point WSAPoll could not be located in the dynamic link library WS2_32.DLL"
The cais Solution¶
cais uses dynamic runtime probing:
1. GetModuleHandleA("ws2_32.dll") is called.
2. GetProcAddress(hws2, "WSAPoll") queries the function pointer.
3. cais clones the exact memory ABI in cais_wsapollfd_t.
4. If found (Vista..11), WSAPoll is invoked.
5. If NULL (NT 3.51..XP), execution transparently routes to cais_poll_via_select().
3. AmigaOS: Sockets as Exec Signals¶
On Commodore Amiga (AmigaOS 3.x/4.x), networking is provided by bsdsocket.library (AmiTCP, Roadshow, Miami).
AmigaOS is not POSIX and lacks hardware interrupt signals like SIGALRM or SIGIO. Instead, multitasking is driven by the Exec microkernel Signal System:
- Each Amiga task has 32 signal bits allocated.
- When a network socket receives data,
bsdsocket.libraryasserts an Exec Signal bit assigned to that process. - Multiplexing is implemented via
WaitSelect(), which maps BSDselect()descriptor masks to Exec signal masks. - The Gotcha: Trying to include
<poll.h>or invokepoll(2)on AmigaOS will fail to compile.caisrecognizesCAIS_IS_AMIGAand keeps network multiplexing directly on the native BSDselect()engine.
4. MS-DOS & WATT-32 Packet Drivers¶
In 16-bit and 32-bit MS-DOS:
- Network cards use Packet Drivers (Class 1 Ethernet interface, hooked onto interrupt vectors between
0x60and0x7F). - There is no preemptive multitasking in the OS; background packet processing occurs during BIOS timer ticks (
INT 08h/INT 1Ch) and serial UART interrupts. - WATT-32 Integration:
caislinks with Waterloo TCP/IP (WATT-32). In non-blocking loops,select_sregularly callsdbug_yield()to allow DOS packet drivers to service the NIC ring buffers.
5. Linux Landlock LSM vs OpenBSD Pledge¶
Kernel-level sandboxing models differ radically across operating systems:
| Feature | Linux Landlock | Apple Darwin Seatbelt | Windows NT Mitigations | OpenBSD Pledge / Unveil | FreeBSD Capsicum |
|---|---|---|---|---|---|
| Enforcement Level | LSM (Linux Security Module) | Kernel MAC Framework (Seatbelt) | Process Mitigation Policies & SRM | Kernel Syscall Filters | Capability Mode |
| Minimum OS Version | Linux 5.13 (2021) | macOS 10.5+ | Windows 8+ (NT 6.2+) | OpenBSD 5.9 (Pledge) / 6.4 (Unveil) | FreeBSD 10.0 |
| Model | Filesystem Path & TCP Restriction | SBPL Profile Language | Dynamic Code & Child Process Rules | Syscall Allowlist + Path Restriction | File Descriptor Rights |
| Inheritance | Strictly inherited by fork() and clone() |
Inherited by child processes | Inherited across child creation | Inherited by fork(), dropped by execve() |
Inherited across descriptors |
cais bridges these paradigms into a uniform API: cais_sandbox_apply() compiles a declarative policy (cais_sandbox_policy_t) into the native kernel primitive of each operating system.