SIL: System Interface Library for games (version 0.5e) ======================================= Copyright (c) 2007-2020 Andrew Church See the file "COPYING.txt" for conditions on use and redistribution. Overview -------- SIL provides a simple, fairly low-level, cross-platform interface covering typical data operations performed by game software: - audio output - graphics rendering - input (joystick/gamepad, keyboard, mouse, touch) - persistent data storage (settings, game progress, achievements) - resource management - threads and synchronization SIL is not a self-contained engine; rather, it is designed to serve as the foundation for such an engine, allowing the engine to be ported more easily to multiple platforms. The API is intended to provide all services needed by a client program, such that client code does not need to make any system calls or use other platform-specific services. SIL is written to the C99 standard (with a few bits in C++, Objective C for Mac OS X and iOS, and Java for Android) and should compile with any standards-compliant compiler. The build system is written to work with GNU Make 3.81 or later, though it is of course possible to use other build systems as well, and the sample SIL program found in the "sample" directory includes an Xcode project that can be used to build and run the SIL test suite on an iOS device. Note that SIL was designed primarily for 2D games. While it is possible to render simple 3D scenes, SIL does not currently have the infrastructure to support complex 3D rendering as can be done with full-fledged 3D engines such as Unreal or Unity. Supported platforms ------------------- SIL includes code for running on the following platforms; each platform has particular build requirements, as documented below. Where a description indicates that "by default" programs will run on a specific version of the platform, the variable TARGET_OS_VERSION can be set in the project's Makefile to explicitly require a newer platform version; see the documentation in build/*/build.mk for details. For details about supported features and library usage on each platform, see the platform-specific README file. - Android: Requires the Android 6.0 SDK (SDK version 23) or later, version 19 or later of the platform-tools and build-tools packages, and version 14 or later of the NDK. Joystick (including the Xperia Play built-in control pad), keyboard (external and on-screen), and touch input are supported. By default, programs will run on any device with Android 2.3.3 or later which supports OpenGL ES 2.0; note that ARM devices require a CPU supporting at least the ARMv7a instruction set. - iOS (and tvOS): Requires Xcode 4.5 or later and version 7.0 or later of the iOS SDK (note that the iOS 7 SDK was first distributed with Xcode 5.0). Touch input, game controller input (for devices running iOS 7.0 or later), and on-screen keyboard input are supported. Game Center support is also available, including isolation of player profiles (such as save files or configuration data) and uploading of achievements to the Game Center server. By default, programs will run on any device with iOS 8.0 or later; SIL also supports iOS versions back to 5.1.1 by setting TARGET_OS_VERSION appropriately, but see README-ios.txt for caveats when targeting such versions. - Linux: Requires X11 and OpenGL headers and libraries, including the X11 extensions XInput2, Xinerama, XRandR, and XF86VidMode (these extensions are optional at runtime, but the headers must be present when building). Programs will run on any system with an X11 windowing interface and a supported OpenGL implementation (see below). Touch input requires version 2.2 of the XInput2 extension, which is included in version 1.12 and later of the X.org X11 server. - Mac OS X: Requires Xcode 4.5 or later and version 10.9 or later of the OS X SDK. By default, programs will run on OS X version 10.7.3 or later. If you get errors like "stdio.h not found" when building tool programs on OS X 10.8 or later, try reinstalling Xcode. If that doesn't help, you may need to run the following command: sudo ln -fs $(xcrun --show-sdk-path)/usr/include /usr/ - Sony PlayStation Portable (PSP): Requires the unofficial PSP SDK and development toolchain from ps2dev.org. Gentoo Linux users can install the SDK and toolchain using a Portage overlay; see . - Windows: Requires a POSIX shell environment (see README-windows.txt) and GNU Make; when building with GCC, requires GCC 4.8.3 or later. Programs will run on Windows XP SP3 and all later versions of Windows with a supported OpenGL implementation (see below) or DirectX 11 (see README-windows.txt for caveats), though horizontal-scroll mouse wheels are not supported by Windows XP and touch input is not supported before Windows 8. On PC-type platforms (Linux, Mac, and Windows), SIL requires an OpenGL implementation which supports OpenGL 2.0 or later or (on Windows only) Direct3D 11 feature level 9_1 or later. As of 2018, the number of PC-type devices in current use (and on which SIL programs would be run -- excluding servers, for example) which do _not_ support at least one of these requirements is believed to be negligible. Note that some older graphics hardware, reportedly including chipsets made by ATI/AMD in the late 2000s, may claim to support OpenGL 2.0 but fail to support non-power-of-two textures (for which support is mandated by OpenGL 2.0); SIL programs which use NPOT textures will not work correctly on systems using such graphics hardware. SIL itself does not require NPOT texture support, though some of the built-in tests (see below) assume support for NPOT textures with no mipmaps and no texture coordinate wrapping, as in OpenGL ES 2.0. On platforms using the 32-bit x86 architecture, SIL assumes the presence of at least SSE2 extensions. SSE2 was introduced in 2001 with the Pentium 4, so this requirement should not be a problem in practice. Structure --------- SIL................ Top-level directory. +-- build.......... Build system for SIL (see "Building a SIL-based | program" below). +-- external....... Source code for independent libraries used by SIL. | See external/README.txt for details. +-- include/SIL.... Header files containing public definitions and | | declarations for use by SIL client code. | +-- configs.... Header files containing common definitions for each | | supported system. | +-- math....... Header files for vector and matrix data types. | +-- resource... Header files for less commonly used resource-related | | functions. | +-- sound...... Header files for less commonly used sound-related | | functions. | +-- sysdep..... Header files for specific systems. The common.h | | header in each directory (if present) augments | | with system-dependent definitions | | and is included automatically with . | +-- utility.... Header files for utility functions. +-- sample......... A simple example demonstrating how to build a | SIL-based program. +-- src............ Source code for SIL. This directory includes source | | files for top-level functionality. | +-- font....... Font management and text rendering functionality. | +-- graphics... Graphics-related functionality, excluding text | | rendering and texture management. | +-- math....... Common mathematical functions. | +-- movie...... Movie playback functions. | +-- resource... Resource (data file) management, including package | | file support. | +-- sound...... Sound channel control and decoding/filtering | | logic, as well as a software mixer for systems | | without (or with insufficient) hardware mixing | | capabilities. | +-- sysdep..... System-dependent code. Each subdirectory of this | | | directory includes code for a specific system | | | (or technology, such as OpenGL) which interacts | | | directly with the host environment. | | +-- misc... Miscellaneous system-level sources shared by | | multiple systems. | +-- test....... Test routines (see "Built-in tests" below). | +-- utility.... Miscellaneous utility routines. +-- testdata....... Data files for the test routines. +-- tools.......... Tool programs useful for manipulating data files. Building a SIL-based program ---------------------------- SIL includes a Makefile-based build system which is designed to simplify the process of building a SIL-based program on multiple platforms. All of the logic for performing the build, as well as auxiliary files used by each platform, can be found in the "build" directory; depending on program complexity, the program's own build script may be as simple as a Makefile listing the program's source files and including "build.mk" from the appropriate platform's build subdirectory. The build system includes a number of configurable options, some applicable to multiple platforms and some specific to particular platforms. See build/common/config.mk and build/*/build.mk for details. The build system presumes that the SIL source tree is located within the overall project tree. It is possible to use the SIL build system with a copy of SIL located outside the project tree by setting the TOPDIR variable to the system's root directory (or some other common parent directory), but this may result in unnecessarily verbose filenames in log messages and other pathname-related inconveniences when debugging. SIL includes copies of several external data processing libraries: FreeType, libnogg, libpng, libvpx, libwebmdec, nestegg, and zlib. By default, these copies of the libraries will be built directly into the final executable if the associated functionality is enabled, but it is possible to instead use an external version (such as a system-wide shared library) by setting the appropriate SIL_LIBRARY_INTERNAL_* configuration variable to 0 in the build configuration. Note that these external libraries have their own licensing terms, separate from SIL; see the file "external/README.txt" for details about each library's license and distribution source. Built-in tests -------------- SIL includes a number of self-test routines which are enabled by setting the SIL_INCLUDE_TESTS Makefile symbol to 1. The tests cover all code in the library except: - Platform-specific I/O functionality, such as hardware sound output, which cannot be tested programmatically. (However, the ALSA-based Linux sound driver is tested using the ALSA loopback plugin if that plugin is available on the system.) - Handling for system errors in platform-specific code. (Error handling in platform-independent code is tested through failure injection.) Current test coverage is approximately 83% by number of executable lines, including 100% of platform-independent code, 94% of OpenGL code, and 74% of other platform-specific code. (Coverage statistics for code executed on Android, iOS, and Mac OS X are inaccurate due to bugs in the Clang toolchain; see note below.) The tests can be run by building and executing the included sample program (in the "sample" directory) as follows: - On systems with command line support (Linux, Mac, Windows), build the program by running "make" in the sample/build directory, then pass the the command-line option "-test" to the program. The program will exit with a successful status if all tests succeed, or an error status if any tests fail. It is also possible to run just a subset of tests with the option "-test=name1,name2,..." (see the definition of tests[] in src/test/test-harness.c for a list of valid test names). - The PSP build also uses a "-test" command-line option, and thus tests can only be run from the PSPlink debugging shell. Use a command line like "program.prx -test" to run the tests. - For Android, if SIL_INCLUDE_TESTS is defined at the Makefile level (as is the case for the sample program), the package will include an extra activity for running tests. Running "make test" for an Android build will start this activity instead of the default activity (which runs the program normally). There is currently no way to select only a subset of tests for running. - For iOS, since there is no way to pass options at runtime, a separate preprocessor macro (#define RUN_TESTS) is used to control whether the tests are run. If both SIL_INCLUDE_TESTS and RUN_TESTS are defined, the executable will run tests and exit instead of running the program normally. RUN_TESTS can be enabled by passing "RUN_TESTS=1" on the "make" command line. There is currently no way to select only a subset of tests for running. The tests include many intentional errors, such as invalid calling patterns and references to nonexistent files, which will generate warning messages via the DLOG() debug logging facility. A summary of test results as well as any failure messages will be output using DLOG() when the tests complete, so that you do not need to scroll back through the log output to find test failures. For example: src/test/test-harness.c:520(show_results): ======== TEST RESULTS ======== src/test/test-harness.c:522(show_results): All tests passed. src/test/test-harness.c:537(show_results): ============================== or src/test/test-harness.c:520(show_results): ======== TEST RESULTS ======== src/test/test-harness.c:528(show_results): passed: condvar [...] src/test/test-harness.c:532(show_results): [*] FAILED: resource_core src/test/test-harness.c:530(show_results): [*] skipped: resource_pkg [...] src/test/test-harness.c:537(show_results): ============================== src/test/test-harness.c:539(show_results): Failure log follows: src/test/resource/core.c:123(test_foo): FAIL: this test failed Additionally, if any tests fail (such as "resource_core" in the above failure example), tests which depend on them (such as "resource_pkg" in the above example) will not be run, in order to avoid reporting multiple failures all caused by the same bug. When running individual tests, the dependencies for a test can be suppressed by prefixing an "=" to the test name; for example, "-test==test1,=test2" will run the "test1" and "test2" tests but not any tests on which they depend. On PC-type platforms, some of the input tests can be affected by real input such as mouse movements, and some mouse-related tests deliberately change the position of the system mouse pointer. The tests make an effort to isolate themselves from the rest of the system, but if (for example) you are actively moving the mouse during the platform-specific input tests, some of the tests may see more input events than they expected and thus fail even though the code itself is correct, or other programs may react in unexpected ways to the synthetic mouse movements. Coverage analysis ----------------- The build control files (build/*/build.mk) also include rules to build the program for coverage analysis. These are primarily intended for testing coverage of SIL itself using the included sample program, but they can also be used to measure coverage of client code. The process for performing coverage analysis varies by system: - Android: Use "make COVERAGE=1" to build for coverage analysis, then install ("make install") and run ("make test") as usual, or use "make all-test COVERAGE=1" to do all three steps at once. After a run, use "make pull-coverage" (with the device connected) to retrieve the coverage data and write results to coverage.out in the build directory. To ensure previous runs do not contaminate the results, run "adb shell rm -r /mnt/sdcard/coverage" before each run. You can change the path to which coverage data is written by specifying GCOV_PREFIX=/absolute/path on the build command line. Note that llvm-cov must be installed and in the shell's executable path on the build system, since it is no longer distributed as part of the NDK. - iOS: Use "make COVERAGE=1 RUN_TESTS=1" to build for testing with coverage analysis, then install and run the program on a device. When the program completes, use Xcode to download the application data directory; the coverage data files will be found under the "Library/Application Support" directory, using the same path structure as on the build system. Find the top-level object directory (normally "obj" or "obj-" under the build directory) in that directory tree, merge its contents into the build tree (using "cp -a SRC/build/obj DEST/build/", for example), then run "make gen-coverage" to generate a coverage.out file in the build directory containing the coverage results (append OBJDIR="obj-" for a multi-architecture build). Note that coverage analysis requires Xcode 7 or later. - Linux: Use "make test-coverage" to build the program, run tests, and write coverage results to coverage.out in the build directory. For the sample program, you can also run "make coverage" to automatically generate HTML output from the coverage results (see below). - Mac OS X: Identical to Linux. Note that coverage analysis requires Xcode 7 or later. - PSP: Use "make COVERAGE=1" to build for coverage analysis, then run the executable on a device. On exit, the executable will write coverage data (*.gcda) files to the same absolute pathnames on the filesystem device under which the executable is running: for example, if the absolute path to the program's build directory is "/home/user/project/build/psp" and the executable is run from the "host0:" device on the PSP, the *.gcda files will be written under "host0:/home/user/project/build/psp". Move these files to the same pathnames on the build filesystem (so the *.gcda files end up in the same directories as the corresponding *.gcno files), then run "make gen-coverage" to generate coverage.out in the build directory. To ensure previous runs do not contaminate the results, remove the directory tree containing the *.gcda files on the PSP filesystem before each run. - Windows: Identical to Linux. In all cases, coverage and non-coverage object files share the same output directory, so be sure to run "make clean" before and after coverage testing. The coverage.out file generated by these build rules can be passed to the tools/cov-html.pl script on that file to create a tree of HTML files showing line and branch coverage for each source file. By default, the HTML file tree is written to a "coverage" subdirectory in the current directory; use the --output-dir option to change the destination path. On PC-type systems, running "make coverage" for the sample program will execute this step automatically, writing the HTML tree to the directory "sample/build/coverage". The HTML generation step can also be run separately with "make coverage-html" in the sample/build directory. If you run the same program on two or more platforms, you can merge the coverage.out files from each platform by passing them through the tools/cov-merge.pl script and sending its output to tools/cov-html.pl. The input files must all be for executables built from the same source code; cov-merge.pl will abort with an error if it finds any differences. Note that branch counts for code executed on multiple systems will be inflated, since each branch path is counted once for each system. In particular, this implies that when a branch path can only be taken on some systems and not on others, a combined analysis will never report 100% branch coverage for that branch even if all branch paths are in fact covered. Source lines are not overcounted in this way, and the line counts will be accurate except in that different compilers may assign different line numbers to the same piece of code -- for example, one compiler may use the closing brace of a loop block for the loop termination branch, while another compiler may use the loop statement for the same branch. Also note that the Clang compiler and associated LLVM tools currently (as of version 3.5) do not seem to produce accurate output; for example, they sometimes claim that comment lines were executed or missed. This particularly affects the iOS and Mac OS X platforms, for which Clang is the only available compiler. Tool programs ------------- SIL includes several tool programs in the "tools" directory which can be useful in creating or converting data files. Notable among these are the "pngtotex" program, which converts a PNG image to the optimized TEX format used by SIL for textures, and the "build-pkg" program, which generates a data package file from which resources can be efficiently loaded. Documentation for each program can be found in comments at the top of the program's source code or by passing the "-h" command line option to the program. Some of the tool programs require external libraries to be installed on the build system: "build-pkg" and "extract-pkg" require zlib, and "pngtotex" requires libpng. If you get link errors when trying to compile the programs, consult your system's documentation for how to install these libraries, or build and install them yourself from the relevant library source code packages. (SIL includes the source code for these libraries under the "external" directory, but those copies are intended only for building into SIL client programs and they have been stripped of some build-related files, so they may not be suitable for independent library builds.) Additionally, when converting textures to compressed data formats, the pngtotex tool requires external tools to be available on the build system. PVRTC format requires the PVRTexToolCLI tool (version 3.40 or any later version with a compatible command-line interface) from the PowerVR SDK distributed by ImgTec; S3TC (DXTn) format requires a custom "dxtcomp" script as described in the pngtotex documentation (see the pngtotex.c source file for details).