222 lines
12 KiB
Markdown
222 lines
12 KiB
Markdown
<div align="center">
|
|
<img src="res/icon.png" alt="Logo" width="auto" height="80">
|
|
|
|
# Volkan
|
|
**C++ Hot-Reload library**
|
|
</div>
|
|
|
|
<br>
|
|
|
|
|
|
A cross-platform, cross-compiler C/C++ library that patches a running
|
|
process's own code in place, so a new build of a function takes effect
|
|
immediately — no restart, and every existing caller (including a raw C
|
|
function pointer cached somewhere) picks it up automatically.
|
|
|
|
```cpp
|
|
// host.cpp -- loads a plugin, calls into it, then hot-reloads it later
|
|
HotReloadContext* ctx = hot_reload_context_create(nullptr);
|
|
hot_reload_load(ctx, "plugin", "plugin.so");
|
|
|
|
auto tick = reinterpret_cast<int (*)(int)>(hot_reload_get_symbol(ctx, "plugin", "tick"));
|
|
tick(1); // runs V1's code
|
|
|
|
hot_reload_reload(ctx, "plugin", "plugin.so"); // plugin.so was rebuilt in between
|
|
tick(1); // same pointer, now runs V2's code
|
|
```
|
|
|
|
## Features
|
|
|
|
- **Works on a real function pointer, not just a lookup table.** Patching
|
|
happens at the function's own address, so *every* path that reaches it
|
|
— a direct call, a call through the PLT/GOT, a raw function pointer
|
|
cached in a struct, a C++ vtable slot — picks up the new code, with
|
|
nothing to re-resolve.
|
|
- **Only executable code is ever touched — mechanically, not by
|
|
convention.** A symbol is a patch candidate only if the loader's own
|
|
metadata says it's a function in an executable section (ELF
|
|
`STT_FUNC`+`SHF_EXECINSTR`, or a PE export in an executable section).
|
|
Data — globals, class instances, vtables, RTTI — is structurally
|
|
excluded, never something you have to remember to avoid exporting.
|
|
- **Deletion is handled, not just tolerated.** A function missing from a
|
|
new build is reported through an event callback, and by default its old
|
|
implementation just keeps running — never a crash, never a jump to
|
|
garbage. Redirect it to your own stub instead if you want stale calls
|
|
to be loud during development.
|
|
- **Real C++ works, not just `extern "C"`.** A real, unmodified mangled
|
|
name — an ordinary method, a virtual method (redirected wherever it's
|
|
reached from, direct call or vtable slot, since both hold the same
|
|
address), an overloaded operator — patches exactly like a plain C
|
|
function, with no special handling needed. See
|
|
["Hot reloading C++ symbols"](source/HotReload/README.md#hot-reloading-c-symbols-classes-virtual-methods-operators)
|
|
for the one real limit this runs into (inlining) and how to work with it.
|
|
- **Failures are diagnosable, not a black box.** `hot_reload_last_error()`
|
|
returns the platform's own explanation (`dlerror()` / a formatted
|
|
`GetLastError()`) for why a load failed — a missing file, a permissions
|
|
problem, and a shared library with a genuinely undefined symbol all look
|
|
identical from the status code alone, and don't need to. A corrupted or
|
|
truncated module is handled with the same care: every offset the ELF
|
|
reader trusts is bounds-checked against the file's actual size, confirmed
|
|
by feeding it 1500 randomly corrupted variants of a real plugin — see
|
|
"What's actually been tested" below.
|
|
- **x86, x86-64, and ARM64, across GCC, Clang, MinGW, and MSVC.** See
|
|
"What's actually been tested" below for exactly how each combination was
|
|
verified — natively, under QEMU, or under Wine, not just reasoned about.
|
|
|
|
## Documentation
|
|
|
|
- **[`source/HotReload/README.md`](source/HotReload/README.md)** is the
|
|
full reference: exactly how patching works, both safety guarantees in
|
|
detail, the complete C++ interoperability story, per-platform scope and
|
|
how each was verified, and a growing list of real bugs found (and
|
|
fixed) by actually running this — on Windows/MSVC, under fuzzing, and
|
|
on every other toolchain in the table below — read this before relying
|
|
on any specific edge case.
|
|
- **Every source file, not just the header, is Doxygen-documented** —
|
|
`hot_reload.h`'s public API (every function, status code, and event
|
|
kind has a `@brief`/`@param`/`@return`) and every internal class and
|
|
helper behind it, down to why each one exists. Build it locally:
|
|
```sh
|
|
scripts/setup.sh # one-time: fetches a pinned Doxygen
|
|
scripts/docs.sh hotreload_lib
|
|
```
|
|
then open `out/docs/hotreload_lib/html/index.html`. Verified to generate
|
|
with zero warnings — for every module this project documents, library,
|
|
examples, and tests alike — even reconfigured with
|
|
`-DDOXYGEN_WARN_IF_UNDOCUMENTED=YES -DDOXYGEN_EXTRACT_PRIVATE=YES`, the
|
|
strictest combination `cmake/Documentation.cmake` exposes.
|
|
- **[`CONTRIBUTING.md`](CONTRIBUTING.md)** covers the CMake build system
|
|
this project itself uses (module conventions, `scripts/`, output
|
|
layout) — only relevant if you're extending *this* repository, not for
|
|
using the library in your own.
|
|
|
|
## How to use it
|
|
|
|
### Build it
|
|
|
|
```sh
|
|
scripts/build.sh # library only
|
|
scripts/build.sh Release -DBUILD_EXAMPLES=ON -DBUILD_TESTS=ON # + both demos + the test suite
|
|
```
|
|
```bat
|
|
scripts\build.bat
|
|
scripts\build.bat Release -DBUILD_EXAMPLES=ON -DBUILD_TESTS=ON
|
|
```
|
|
|
|
This produces `hotreload_lib`, a `STATIC` library with a plain C API
|
|
(`hot_reload.h`) over a C++ implementation. Link your host application
|
|
against it (`target_link_libraries(your_app PRIVATE hotreload_lib)` if
|
|
you're using CMake; see [`source/HotReload/CMakeLists.txt`](source/HotReload/CMakeLists.txt)
|
|
for what it links on each platform if you're not).
|
|
|
|
### See it work — a scripted, deterministic demo
|
|
|
|
```sh
|
|
./out/build/bin/examples/hotreload_demo_host
|
|
```
|
|
|
|
Loads one plugin, calls into it, hot-reloads a second build of it, and
|
|
calls the *same, never-re-resolved* function pointers again — proving
|
|
every guarantee above in one run. Open
|
|
[`source/examples/HotReloadDemo/PluginV1/src/Plugin.cpp`](source/examples/HotReloadDemo/PluginV1/src/Plugin.cpp)
|
|
and its `PluginV2` counterpart side by side against the output.
|
|
|
|
### Try it yourself — a persistent, interactive demo
|
|
|
|
```sh
|
|
./out/build/bin/examples/hotreload_live_demo_host # leave this running
|
|
cmake --build out/build --target hotreload_live_plugin # in another terminal, or:
|
|
source/examples/HotReloadLiveDemo/watch.sh out/build # rebuild automatically on save
|
|
```
|
|
|
|
Edit [`source/examples/HotReloadLiveDemo/Plugin/src/LivePlugin.cpp`](source/examples/HotReloadLiveDemo/Plugin/src/LivePlugin.cpp)
|
|
— its own comment walks through several concrete things worth trying —
|
|
and watch the *running* process pick up your change within about a
|
|
second, with no restart. See
|
|
[`source/examples/HotReloadLiveDemo/README.md`](source/examples/HotReloadLiveDemo/README.md)
|
|
for the full walkthrough, including two Windows-specific gotchas
|
|
(`Access is denied` on "Rebuild", and why leftover `.hotreload.N` files
|
|
next to your plugin are expected, not a leak) that are easy to mistake
|
|
for bugs the first time you hit them.
|
|
|
|
### Use it in your own module
|
|
|
|
1. `extern "C"` isn't required — a real C++ mangled name works too — but
|
|
it's the simplest choice for anything you want to look up by a plain,
|
|
human-typed name via `hot_reload_get_symbol()`.
|
|
2. Call `hot_reload_enable_patchable_functions(<target>)`
|
|
(`cmake/HotReload.cmake`) on your plugin's CMake target, so every
|
|
exported function has guaranteed room for a jump.
|
|
3. Optionally call `hot_reload_restrict_exports(<target>)` and mark just
|
|
your intended entry points with `HOT_RELOAD_EXPORT`, instead of
|
|
exporting everything by default — keeps your own internal helpers out
|
|
of the reloadable surface. See `source/HotReload/README.md`'s "Using
|
|
it in your own module" for the full version of this list, including
|
|
why this specific step doesn't (and can't) quiet `std::string`/
|
|
`std::vector` noise, and why that's fine.
|
|
4. Have your host `hot_reload_load()` it once at startup and
|
|
`hot_reload_reload()` it after every rebuild, from a point where
|
|
nothing else is actively calling into it. A path with no `/` in it is
|
|
still resolved against the current working directory (never a system
|
|
library path), so a plain `plugin.so` next to your host works exactly
|
|
as you'd expect.
|
|
|
|
## Limitations
|
|
|
|
Two things this library fundamentally cannot detect, because a symbol
|
|
table doesn't carry the information needed to:
|
|
|
|
- **A changed function signature.** Patching redirects control flow at a
|
|
fixed address; it has no way to know that `void tick(int)` became
|
|
`void tick(int, int)` between builds. Give a changed signature a new
|
|
exported name instead of reusing the old one.
|
|
- **A data layout change reached through code that still gets patched.**
|
|
This library guarantees it never *touches* a data symbol directly. It
|
|
cannot guarantee that a patched function is safe to run against an
|
|
object that already exists in memory with the *old* layout — e.g. a
|
|
C++ vtable gaining a new virtual function, or a struct gaining a field,
|
|
while an old instance is still alive.
|
|
|
|
One thing it doesn't fully solve, by design rather than by omission:
|
|
calling `hot_reload_reload()` **concurrently with active execution** of
|
|
the code being replaced isn't guaranteed safe on every architecture — the
|
|
intended pattern (shared by every tool in this space) is reloading from a
|
|
quiescent point, such as your main thread between frames. See
|
|
`source/HotReload/README.md`'s "What this cannot detect" for the full
|
|
reasoning, including exactly which write-ordering protections *are* in
|
|
place and what they do and don't cover.
|
|
|
|
## What's actually been tested
|
|
|
|
Not just built for, actually run to completion, verified directly rather
|
|
than assumed:
|
|
|
|
| Toolchain | How it was verified |
|
|
|---|---|
|
|
| GCC + Clang, native Linux/x86-64 | Both compilers, full demo suite and test suite run to completion, including a CET (`endbr64`) landing pad this toolchain turns out to emit by default, and again explicitly forcing one with `-fcf-protection=full` |
|
|
| GCC, native Linux/x86 (32-bit) | Full demo and test suite run to completion under `-m32`. This is also where the suite's own "too small to patch" plugin function turned out not to be small enough at `-O2` on this one target — a real, previously-unverified gap between what the test assumed and what the compiler actually emitted, closed by making that function require no instructions at all |
|
|
| ARM64 (GCC cross-compiled) | Run under `qemu-aarch64` — twice, once plain and once with `-mbranch-protection=standard`, to exercise the BTI landing-pad path on real ARM64 semantics |
|
|
| MinGW | Cross-compiled and run under Wine, including the PE export-table parser and a real `LoadLibrary` failure (confirmed `hot_reload_last_error()` reports "Bad EXE format" correctly) |
|
|
| MSVC | Built and live-tested on real Windows by someone using this library — not by the environment that wrote it. Four real fixes came out of that and are folded in: `windows.h`/`dbghelp.h` include order, `NOMINMAX`, explicit `dbghelp` linking, and a reload-retry widening for Windows' slower post-rebuild file-availability timing |
|
|
| AddressSanitizer + UndefinedBehaviorSanitizer, and Valgrind with self-modifying-code checking enabled | Full test suite, clean under all three — the self-modifying-code checking matters specifically here, since patching a function's own bytes at runtime is exactly the pattern a memory checker's JIT otherwise mishandles by default |
|
|
| Fuzzing (ELF section-header parsing) | 1500 randomly corrupted variants of a real plugin's section header table — each still a file the OS loader itself accepts via `dlopen()` — all handled cleanly. An earlier version of this same reader crashed on 69 of an initial 300 |
|
|
|
|
Three more real, reproducible bugs came out of this same pass, each
|
|
confirmed broken before the fix and confirmed fixed after:
|
|
a `hot_reload_load()`/`hot_reload_reload()` path with no `/` in it (e.g.
|
|
a plain `"plugin.so"`) silently failed to load on POSIX, since `dlopen()`
|
|
treats a bare name as a library-search request rather than a file path;
|
|
a module name containing a `/` or other unusual character (the header
|
|
never promised it had to look like a file name) broke the internal
|
|
staging path used to make each reload load from a fresh copy; and the
|
|
ELF section-header hardening above.
|
|
|
|
**Deliberately not implemented, rather than implemented and untested:**
|
|
32-bit ARM (its ARM/Thumb instruction-mode ambiguity fails silently if
|
|
gotten wrong, and there was no way to verify it on real or emulated
|
|
hardware) and macOS/Mach-O (never requested, and would need a third image
|
|
format reader alongside the existing ELF and PE ones). Both are
|
|
documented, bounded gaps — see `source/HotReload/README.md`'s "Scope"
|
|
section for the reasoning behind each, and the extension point if you
|
|
need either.
|