42 Commits
Author SHA1 Message Date
Valentin Popov 162de8ccab fix: require manifests for licensed gates
Docs Deploy / Build and Deploy MkDocs (push) Successful in 35s
Test / Lint (push) Failing after 1m16s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
2026-06-22 17:29:33 +04:00
Valentin Popov 0b23cf48e7 fix: use canonical sha256 world hashes 2026-06-22 17:11:21 +04:00
Valentin Popov 7356238ffb fix: harden render command validation 2026-06-22 17:05:45 +04:00
Valentin Popov 42441082f0 fix: cap fixed-step catch-up 2026-06-22 17:02:00 +04:00
Valentin Popov ccd61c05b0 fix: expose configurable rsli read profiles 2026-06-22 16:58:59 +04:00
Valentin Popov 813beec7be fix: preserve nres gaps during edits 2026-06-22 16:55:10 +04:00
Valentin Popov 91c7a8a14e fix: make corpus reports explicit and fallible 2026-06-22 16:49:32 +04:00
Valentin Popov 8b91a0bfbf fix: make core error displays actionable 2026-06-22 16:41:21 +04:00
Valentin Popov fb97405e0c fix: decode payloads outside resource lock 2026-06-22 16:36:50 +04:00
Valentin Popov d579b696e6 fix: cap decoded payload cache bytes 2026-06-22 16:34:14 +04:00
Valentin Popov aa1b809bd8 fix: strengthen resource fingerprints 2026-06-22 16:31:57 +04:00
Valentin Popov f69c893a40 fix: harden path lookup and mark gl backend gap 2026-06-22 16:12:57 +04:00
Valentin Popov 5436727961 docs: mark stage 4 runtime gaps explicit 2026-06-22 16:04:35 +04:00
Valentin Popov be41fa839f fix: harden resource and world state correctness 2026-06-22 16:02:16 +04:00
Valentin Popov 8e5e46b7b3 fix: make ci locked and isolate licensed tests 2026-06-22 15:55:37 +04:00
Valentin Popov d0bdbaa1ed feat: implement FParkan architecture foundation
Docs Deploy / Build and Deploy MkDocs (push) Successful in 35s
Test / Lint (push) Failing after 1m14s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
Add the modular fparkan workspace, domain crates, adapters, apps, xtask policy/CI, acceptance evidence, and licensed corpus gates for the macOS-focused roadmap foundation.
2026-06-22 13:13:32 +04:00
Valentin Popov 7416fdc7e9 Merge pull request 'chore(deps): update actions/checkout action to v7' (#16) from renovate/actions-checkout-7.x into devel
Docs Deploy / Build and Deploy MkDocs (push) Successful in 34s
Test / Lint (push) Failing after 1m8s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
Reviewed-on: #16
2026-06-22 03:09:59 +04:00
Valentin Popov 78fc5f1deb docs: rewrite MkDocs documentation
Docs Deploy / Build and Deploy MkDocs (push) Successful in 34s
Test / Lint (push) Failing after 1m7s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
2026-06-22 01:58:51 +04:00
Valentin Popov 50c2cf4686 chore: remove Python tooling and resource viewer
Docs Deploy / Build and Deploy MkDocs (push) Successful in 2m6s
Test / Lint (push) Failing after 1m10s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
2026-06-22 00:35:19 +04:00
renovate[bot] a63290fbc8 chore(deps): update actions/checkout action to v7
Test / Lint (push) Failing after 1m2s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
Test / Lint (pull_request) Failing after 1m1s
Test / Test (pull_request) Has been skipped
Test / Render parity (pull_request) Has been skipped
2026-06-19 00:05:00 +00:00
renovate[bot] 96a25b6c0e fix(deps): update rust crate glow to 0.17
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
Test / Lint (pull_request) Failing after 1m1s
Test / Test (pull_request) Has been skipped
Test / Render parity (pull_request) Has been skipped
Docs Deploy / Build and Deploy MkDocs (push) Successful in 46s
Test / Lint (push) Failing after 1m5s
RenovateBot / renovate (push) Successful in 25s
2026-03-08 00:02:26 +00:00
Valentin Popov f4262cf369 Merge pull request 'fix(deps): update rust crate toml to v1' (#14) from renovate/toml-1.x into devel
Docs Deploy / Build and Deploy MkDocs (push) Successful in 33s
Test / Lint (push) Failing after 1m3s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
RenovateBot / renovate (push) Successful in 28s
Reviewed-on: #14
2026-03-02 18:42:03 +04:00
renovate[bot] 9b100b8fc3 chore(deps): update actions/upload-artifact action to v7
Test / Lint (pull_request) Failing after 1m6s
Test / Test (pull_request) Has been skipped
Test / Render parity (pull_request) Has been skipped
Test / Lint (push) Has been cancelled
Test / Test (push) Has been cancelled
Test / Render parity (push) Has been cancelled
Docs Deploy / Build and Deploy MkDocs (push) Has been cancelled
2026-02-27 00:01:47 +00:00
renovate[bot] 9fceeb9a0a fix(deps): update rust crate toml to v1
Test / Lint (push) Failing after 1m2s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
Test / Lint (pull_request) Failing after 1m4s
Test / Test (pull_request) Has been skipped
Test / Render parity (pull_request) Has been skipped
2026-02-26 00:01:14 +00:00
renovate[bot] 4b7f1a16b9 fix(deps): update all digest updates
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
Test / Lint (pull_request) Failing after 59s
Test / Test (pull_request) Has been skipped
Test / Render parity (pull_request) Has been skipped
Docs Deploy / Build and Deploy MkDocs (push) Successful in 32s
Test / Lint (push) Failing after 1m2s
RenovateBot / renovate (push) Successful in 28s
2026-02-25 00:01:21 +00:00
Valentin Popov ada3b903ad chore: update docs deployment branch from master to devel
Docs Deploy / Build and Deploy MkDocs (push) Successful in 1m9s
Test / Lint (push) Failing after 2m11s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
RenovateBot / renovate (push) Successful in 1m7s
2026-02-24 22:42:18 +00:00
Valentin Popov 31d849ddbf updated docs
Test / Lint (push) Failing after 1m57s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
2026-02-19 16:10:57 +04:00
Valentin Popov 4ef08d0bf6 feat: add terrain-core, tma, and unitdat crates with parsing functionality
- Introduced `terrain-core` crate for loading and processing terrain mesh data.
- Added `tma` crate for parsing mission files, including footer and object records.
- Created `unitdat` crate for reading unit data files with validation of structure.
- Implemented error handling and tests for all new crates.
- Documented object registry format and rendering pipeline in specifications.
2026-02-19 16:07:01 +04:00
Valentin Popov 598137ed13 feat(resource-viewer): добавить новый ресурсный просмотрщик с базовой функциональностью
Test / Lint (push) Failing after 2m30s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
feat(nres): улучшить структуру архива с добавлением заголовка и информации о записях
feat(rsli): добавить поддержку заголовка библиотеки и улучшить обработку записей
2026-02-19 10:51:54 +00:00
Valentin Popov cb0ca2f2f0 feat(render-demo): добавить отображение FPS в заголовок окна и stdout в интерактивном режиме
Test / Lint (push) Failing after 1m16s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
2026-02-19 10:27:10 +00:00
Valentin Popov 7346e695c4 feat(render-demo): обновить поддержку OpenGL с добавлением выбора между GLES2 и Core 3.3
Test / Lint (push) Failing after 1m17s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
2026-02-19 10:17:14 +00:00
Valentin Popov bb827c3928 feat: Refactor code structure and enhance functionality across multiple crates 2026-02-19 10:09:18 +00:00
Valentin Popov efab61a45c feat(render-core): add default UV scale and refactor UV mapping logic
Test / Lint (push) Failing after 1m12s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
- Introduced a constant `DEFAULT_UV_SCALE` for UV scaling.
- Refactored UV mapping in `build_render_mesh` to use the new constant.
- Simplified `compute_bounds` functions by extracting common logic into `compute_bounds_impl`.

test(render-core): add tests for rendering with empty and multi-node models

- Added tests to verify behavior when building render meshes from models with no slots and multiple nodes.
- Ensured UV scaling is correctly applied in tests.

feat(render-demo): add FOV argument and improve error handling

- Added a `--fov` command-line argument to set the field of view.
- Enhanced error messages for texture resolution failures.
- Updated MVP computation to use the new FOV parameter.

fix(rsli): improve error handling in LZH decompression

- Added checks to prevent out-of-bounds access in LZH decoding logic.

refactor(texm): streamline texture parsing and decoding tests

- Created a helper function `build_texm_payload` for constructing test payloads.
- Added tests for various texture formats including RGB565, RGB556, ARGB4444, and Luminance Alpha.
- Improved error handling for invalid TEXM headers and mip bounds.
2026-02-19 09:46:23 +00:00
Valentin Popov 0d7ae6a017 Документирование и обновление спецификаций
Test / Lint (push) Failing after 1m10s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
- Обновлены спецификации `runtime-pipeline`, `sound`, `terrain-map-loading`, `texture`, `ui` и `wear`.
- Добавлены разделы о статусе покрытия и оставшихся задачах для достижения 100% завершенности.
- Внесены уточнения по архитектурным ролям, минимальным контрактам и требованиям к toolchain для каждой подсистемы.
- Уточнены форматы данных и правила взаимодействия между компонентами системы.
2026-02-19 11:07:04 +04:00
Valentin Popov a281ffa32e feat: Enhance model and texture loading with improved error handling and new features
Test / Lint (push) Failing after 1m10s
Test / Test (push) Has been skipped
Test / Render parity (push) Has been skipped
- Introduced `LoadedModel` and `LoadedTexture` structs for better encapsulation of model and texture data.
- Added functions to load models and textures from archives, including support for resolving textures based on materials and wear entries.
- Implemented error handling for missing textures, materials, and wear entries.
- Updated the rendering pipeline to support texture loading and binding, including command-line arguments for texture customization.
- Enhanced the `texm` crate with new decoding capabilities for various pixel formats, including indexed textures.
- Added tests for texture decoding and loading to ensure reliability and correctness.
- Updated documentation to reflect changes in the material and texture resolution process.
2026-02-19 05:19:18 +04:00
Valentin Popov 18d4c6cf9f feat(render-parity): add deterministic frame comparison tool
- Introduced `render-parity` crate for comparing rendered frames against reference images.
- Added command-line options for specifying manifest and output directory.
- Implemented image comparison metrics: mean absolute difference, maximum absolute difference, and changed pixel ratio.
- Created a configuration file `cases.toml` for defining test cases with global defaults and specific parameters.
- Added functionality to capture frames from `render-demo` and save diff images on discrepancies.
- Updated documentation to include usage instructions and CI model for automated testing.
2026-02-19 05:02:26 +04:00
Valentin Popov 0e19660eb5 Refactor documentation structure and add new specifications
- Updated MSH documentation to reflect changes in material, wear, and texture specifications.
- Introduced new `render.md` file detailing the render pipeline process.
- Removed outdated sections from `runtime-pipeline.md` and redirected to `render.md`.
- Added detailed specifications for `Texm` texture format and `WEAR` wear table.
- Updated navigation in `mkdocs.yml` to align with new documentation structure.
2026-02-19 04:46:23 +04:00
Valentin Popov 8a69872576 Refactor code structure for improved readability and maintainability
Test / Lint (push) Successful in 1m8s
Test / Test (push) Successful in 1m15s
2026-02-12 11:07:25 +00:00
Valentin Popov aa68906a3d feat: добавить экспорт в формат OBJ для рендеринга террейна и опциональных полигонов 2026-02-12 10:36:15 +00:00
Valentin Popov 8bf3b7b209 feat: добавить 3D рендерер для террейна с поддержкой Land.msh и Land.map 2026-02-12 10:24:42 +00:00
Valentin Popov 669fb40a70 Add terrain map documentation validator
This commit introduces a new Python script, `terrain_map_doc_validator.py`, which validates terrain and map documentation assumptions against actual game data. The validator checks for the presence and correctness of various data chunks in the `Land.msh` and `Land.map` files, reporting any issues found during the validation process. It also generates a summary report of the validation results, including counts of errors and warnings, and statistics related to the map and mesh data.
2026-02-12 10:17:41 +00:00
Valentin Popov 9c0df3d299 feat: добавить скрипт для детерминированного аудита FXID "absolute parity" 2026-02-12 08:22:51 +00:00
148 changed files with 35970 additions and 11325 deletions
+2
View File
@@ -0,0 +1,2 @@
[alias]
xtask = "run -p xtask --"
+2 -2
View File
@@ -3,7 +3,7 @@ name: Docs Deploy
on:
push:
branches:
- master
- devel
jobs:
deploy-docs:
@@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up Python
uses: actions/setup-python@v6
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Run renovate
run: |
+30 -2
View File
@@ -7,7 +7,7 @@ jobs:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
@@ -21,7 +21,35 @@ jobs:
runs-on: ubuntu-latest
needs: lint
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- name: Cargo test
run: cargo test --workspace --all-features -- --nocapture
render-parity:
name: Render parity
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- name: Install headless GL runtime
run: |
sudo apt-get update
sudo apt-get install -y xvfb libgl1-mesa-dri libgles2-mesa-dev mesa-utils
- name: Build render-demo binary
run: cargo build -p render-demo --features demo
- name: Run frame parity suite
run: |
xvfb-run -s "-screen 0 1280x720x24" cargo run -p render-parity -- \
--manifest parity/cases.toml \
--output-dir target/render-parity/current \
--demo-bin target/debug/parkan-render-demo \
--keep-going
- name: Upload parity artifacts
if: always()
uses: actions/upload-artifact@v7
with:
name: render-parity-artifacts
path: target/render-parity/current
if-no-files-found: ignore
-4
View File
@@ -69,10 +69,6 @@ $RECYCLE.BIN/
debug/
target/
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock
# These are backup files generated by rustfmt
**/*.rs.bk
Generated
+315
View File
@@ -0,0 +1,315 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "adler2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "crc32fast"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511"
dependencies = [
"cfg-if",
]
[[package]]
name = "encoding_rs"
version = "0.8.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75030f3c4f45dafd7586dd6780965a8c7e8e285a5ecb86713e63a79c5b2766f3"
dependencies = [
"cfg-if",
]
[[package]]
name = "flate2"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"crc32fast",
"miniz_oxide",
]
[[package]]
name = "fparkan-animation"
version = "0.1.0"
[[package]]
name = "fparkan-assets"
version = "0.1.0"
dependencies = [
"fparkan-material",
"fparkan-msh",
"fparkan-nres",
"fparkan-path",
"fparkan-prototype",
"fparkan-resource",
"fparkan-texm",
"fparkan-vfs",
]
[[package]]
name = "fparkan-binary"
version = "0.1.0"
[[package]]
name = "fparkan-cli"
version = "0.1.0"
dependencies = [
"fparkan-corpus",
"fparkan-nres",
"fparkan-prototype",
"fparkan-resource",
"fparkan-rsli",
"fparkan-runtime",
"fparkan-vfs",
]
[[package]]
name = "fparkan-corpus"
version = "0.1.0"
dependencies = [
"fparkan-binary",
"fparkan-nres",
"fparkan-path",
]
[[package]]
name = "fparkan-diagnostics"
version = "0.1.0"
[[package]]
name = "fparkan-fx"
version = "0.1.0"
dependencies = [
"fparkan-binary",
"fparkan-nres",
]
[[package]]
name = "fparkan-game"
version = "0.1.0"
dependencies = [
"fparkan-render",
"fparkan-runtime",
"fparkan-vfs",
"fparkan-world",
]
[[package]]
name = "fparkan-headless"
version = "0.1.0"
dependencies = [
"fparkan-runtime",
"fparkan-vfs",
"fparkan-world",
]
[[package]]
name = "fparkan-material"
version = "0.1.0"
dependencies = [
"encoding_rs",
"fparkan-nres",
"fparkan-path",
"fparkan-resource",
"fparkan-vfs",
]
[[package]]
name = "fparkan-mission-format"
version = "0.1.0"
dependencies = [
"encoding_rs",
"fparkan-binary",
]
[[package]]
name = "fparkan-msh"
version = "0.1.0"
dependencies = [
"encoding_rs",
"fparkan-animation",
"fparkan-nres",
]
[[package]]
name = "fparkan-nres"
version = "0.1.0"
dependencies = [
"fparkan-binary",
"fparkan-path",
]
[[package]]
name = "fparkan-path"
version = "0.1.0"
[[package]]
name = "fparkan-platform"
version = "0.1.0"
[[package]]
name = "fparkan-platform-sdl"
version = "0.1.0"
dependencies = [
"fparkan-platform",
]
[[package]]
name = "fparkan-prototype"
version = "0.1.0"
dependencies = [
"encoding_rs",
"fparkan-binary",
"fparkan-material",
"fparkan-msh",
"fparkan-nres",
"fparkan-path",
"fparkan-resource",
"fparkan-texm",
"fparkan-vfs",
]
[[package]]
name = "fparkan-render"
version = "0.1.0"
dependencies = [
"fparkan-world",
]
[[package]]
name = "fparkan-render-gl"
version = "0.1.0"
dependencies = [
"fparkan-render",
]
[[package]]
name = "fparkan-resource"
version = "0.1.0"
dependencies = [
"fparkan-binary",
"fparkan-nres",
"fparkan-path",
"fparkan-rsli",
"fparkan-vfs",
]
[[package]]
name = "fparkan-rsli"
version = "0.1.0"
dependencies = [
"flate2",
]
[[package]]
name = "fparkan-runtime"
version = "0.1.0"
dependencies = [
"fparkan-mission-format",
"fparkan-nres",
"fparkan-path",
"fparkan-platform",
"fparkan-prototype",
"fparkan-render",
"fparkan-resource",
"fparkan-terrain",
"fparkan-terrain-format",
"fparkan-vfs",
"fparkan-world",
]
[[package]]
name = "fparkan-terrain"
version = "0.1.0"
dependencies = [
"fparkan-nres",
"fparkan-terrain-format",
]
[[package]]
name = "fparkan-terrain-format"
version = "0.1.0"
dependencies = [
"fparkan-binary",
"fparkan-nres",
]
[[package]]
name = "fparkan-test-support"
version = "0.1.0"
dependencies = [
"fparkan-render",
]
[[package]]
name = "fparkan-texm"
version = "0.1.0"
dependencies = [
"fparkan-nres",
]
[[package]]
name = "fparkan-vfs"
version = "0.1.0"
dependencies = [
"fparkan-binary",
"fparkan-path",
]
[[package]]
name = "fparkan-viewer"
version = "0.1.0"
dependencies = [
"fparkan-msh",
"fparkan-nres",
"fparkan-render",
"fparkan-resource",
"fparkan-rsli",
"fparkan-terrain-format",
"fparkan-texm",
"fparkan-vfs",
]
[[package]]
name = "fparkan-world"
version = "0.1.0"
dependencies = [
"fparkan-binary",
]
[[package]]
name = "miniz_oxide"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "simd-adler32"
version = "0.3.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e320a6c5ad31d271ad523dcf3ad13e2767ad8b1cb8f047f75a8aeaf8da139da2"
[[package]]
name = "xtask"
version = "0.1.0"
dependencies = [
"fparkan-corpus",
]
+57 -1
View File
@@ -1,6 +1,62 @@
[workspace]
resolver = "3"
members = ["crates/*"]
members = [
"crates/fparkan-animation",
"crates/fparkan-assets",
"crates/fparkan-binary",
"crates/fparkan-corpus",
"crates/fparkan-diagnostics",
"crates/fparkan-fx",
"crates/fparkan-material",
"crates/fparkan-mission-format",
"crates/fparkan-msh",
"crates/fparkan-nres",
"crates/fparkan-path",
"crates/fparkan-platform",
"crates/fparkan-prototype",
"crates/fparkan-render",
"crates/fparkan-resource",
"crates/fparkan-rsli",
"crates/fparkan-runtime",
"crates/fparkan-terrain",
"crates/fparkan-terrain-format",
"crates/fparkan-test-support",
"crates/fparkan-texm",
"crates/fparkan-vfs",
"crates/fparkan-world",
"adapters/fparkan-platform-sdl",
"adapters/fparkan-render-gl",
"apps/fparkan-cli",
"apps/fparkan-game",
"apps/fparkan-headless",
"apps/fparkan-viewer",
"xtask",
]
[workspace.package]
version = "0.1.0"
edition = "2021"
license = "GPL-2.0-only"
repository = "https://github.com/valentineus/fparkan"
[workspace.lints.rust]
unsafe_code = "forbid"
missing_docs = "warn"
unreachable_pub = "warn"
unused_must_use = "deny"
[workspace.lints.clippy]
all = { level = "deny", priority = -1 }
pedantic = { level = "warn", priority = -1 }
unwrap_used = "deny"
expect_used = "deny"
panic = "deny"
todo = "deny"
unimplemented = "deny"
dbg_macro = "deny"
print_stdout = "warn"
print_stderr = "warn"
lossy_float_literal = "deny"
[profile.release]
codegen-units = 1
+35 -13
View File
@@ -1,13 +1,12 @@
# FParkan
Open source проект с реализацией компонентов игрового движка игры **«Паркан: Железная Стратегия»** и набором [вспомогательных инструментов](tools) для исследования.
Open source проект с реализацией компонентов игрового движка игры **«Паркан: Железная Стратегия»**.
## Описание
Проект находится в активной разработке и включает:
- библиотеки для работы с форматами игровых архивов;
- инструменты для валидации/подготовки тестовых данных;
- спецификации форматов и сопутствующую документацию.
## Установка
@@ -19,28 +18,51 @@ Open source проект с реализацией компонентов игр
- локально: каталог [`docs/`](docs)
- сайт: <https://fparkan.popov.link>
## Инструменты
Вспомогательные инструменты находятся в каталоге [`tools/`](tools).
- [tools/archive_roundtrip_validator.py](tools/archive_roundtrip_validator.py) — инструмент верификации документации по архивам `NRes`/`RsLi` на реальных файлах (включая `unpack -> repack -> byte-compare`).
- [tools/init_testdata.py](tools/init_testdata.py) — подготовка тестовых данных по сигнатурам с раскладкой по каталогам.
## Библиотеки
- [crates/nres](crates/nres) — библиотека для работы с файлами архивов NRes (чтение, поиск, редактирование, сохранение).
- [crates/rsli](crates/rsli) — библиотека для работы с файлами архивов RsLi (чтение, поиск, загрузка/распаковка поддерживаемых методов).
- [crates/fparkan-nres](crates/fparkan-nres) — strict/lossless модель архивов NRes.
- [crates/fparkan-rsli](crates/fparkan-rsli) — чтение, lookup и lossless roundtrip архивов RsLi.
- [crates/fparkan-msh](crates/fparkan-msh) — validated static MSH geometry.
- [crates/fparkan-runtime](crates/fparkan-runtime) — transactional mission loading и headless runtime foundation.
- [apps/fparkan-cli](apps/fparkan-cli), [apps/fparkan-viewer](apps/fparkan-viewer), [apps/fparkan-headless](apps/fparkan-headless), [apps/fparkan-game](apps/fparkan-game) — composition roots.
## Тестирование
Базовое тестирование проходит на синтетических тестах из репозитория.
Базовое тестирование проходит на синтетических тестах из репозитория:
```bash
cargo xtask ci
```
Для дополнительного тестирования на реальных игровых ресурсах:
- используйте [tools/init_testdata.py](tools/init_testdata.py) для подготовки локального набора;
- используйте оригинальную копию игры (диск или [GOG-версия](https://www.gog.com/en/game/parkan_iron_strategy));
- разместите игровые каталоги в [`testdata/`](testdata);
- игровые ресурсы в репозиторий не включаются, так как защищены авторским правом.
Локальный licensed gate использует некоммитимый manifest:
```bash
cat > /private/tmp/fparkan-corpora.toml <<'EOF'
schema = 1
[[corpus]]
id = "part1-local"
kind = "part1"
root = "/absolute/path/to/IS"
expected_profile = "parkan-is-part1"
[[corpus]]
id = "part2-local"
kind = "part2"
root = "/absolute/path/to/IS2"
expected_profile = "parkan-is-part2"
EOF
FPARKAN_CORPORA_MANIFEST=/private/tmp/fparkan-corpora.toml \
cargo xtask acceptance report --suite licensed --stage 5
```
## Contributing & Support
Проект активно поддерживается и открыт для contribution. Issues и pull requests можно создавать в обоих репозиториях:
+12
View File
@@ -0,0 +1,12 @@
[package]
name = "fparkan-platform-sdl"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-platform = { path = "../../crates/fparkan-platform" }
[lints]
workspace = true
+123
View File
@@ -0,0 +1,123 @@
#![forbid(unsafe_code)]
//! SDL platform adapter boundary stubs behind safe `FParkan` ports.
use fparkan_platform::{
EventSource, GraphicsContextRequest, GraphicsProfile, PhysicalSize, PlatformError,
PlatformEvent, Version, WindowPort,
};
/// Adapter capabilities compiled into this package.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct SdlAdapterCapabilities {
/// Supported graphics context requests in preference order.
pub graphics: Vec<GraphicsContextRequest>,
/// Whether adapter-owned code is free of `unsafe`.
pub project_owned_unsafe_free: bool,
}
impl Default for SdlAdapterCapabilities {
fn default() -> Self {
Self {
graphics: vec![
GraphicsContextRequest {
profile: GraphicsProfile::DesktopCore,
version: Version { major: 3, minor: 3 },
},
GraphicsContextRequest {
profile: GraphicsProfile::Embedded,
version: Version { major: 2, minor: 0 },
},
],
project_owned_unsafe_free: true,
}
}
}
/// Returns whether the project-owned adapter boundary avoids `unsafe`.
#[must_use]
pub fn project_owned_layer_unsafe_free() -> bool {
SdlAdapterCapabilities::default().project_owned_unsafe_free
}
/// In-memory event source used by adapter smoke tests before a concrete SDL
/// runtime is selected.
#[derive(Clone, Debug, Default)]
pub struct SdlEventSourceStub {
pending: Vec<PlatformEvent>,
}
impl SdlEventSourceStub {
/// Creates an event source with deterministic pending events.
#[must_use]
pub fn new(pending: Vec<PlatformEvent>) -> Self {
Self { pending }
}
}
impl EventSource for SdlEventSourceStub {
fn poll(&mut self, out: &mut Vec<PlatformEvent>) -> Result<(), PlatformError> {
out.append(&mut self.pending);
Ok(())
}
}
/// Safe window-port stub with SDL-compatible drawable-size semantics.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct SdlWindowStub {
size: PhysicalSize,
presents: u64,
}
impl SdlWindowStub {
/// Creates a stub window with a fixed drawable size.
#[must_use]
pub fn new(size: PhysicalSize) -> Self {
Self { size, presents: 0 }
}
/// Number of successful present calls.
#[must_use]
pub fn presents(&self) -> u64 {
self.presents
}
}
impl WindowPort for SdlWindowStub {
fn drawable_size(&self) -> PhysicalSize {
self.size
}
fn present(&mut self) -> Result<(), PlatformError> {
self.presents = self.presents.saturating_add(1);
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn adapter_boundary_is_project_owned_unsafe_free() {
assert!(project_owned_layer_unsafe_free());
assert_eq!(SdlAdapterCapabilities::default().graphics.len(), 2);
}
#[test]
fn event_source_and_window_ports_are_deterministic() -> Result<(), PlatformError> {
let mut source = SdlEventSourceStub::new(vec![PlatformEvent::Quit]);
let mut events = Vec::new();
source.poll(&mut events)?;
source.poll(&mut events)?;
assert_eq!(events, vec![PlatformEvent::Quit]);
let mut window = SdlWindowStub::new(PhysicalSize {
width: 320,
height: 240,
});
assert_eq!(window.drawable_size().width, 320);
window.present()?;
assert_eq!(window.presents(), 1);
Ok(())
}
}
+12
View File
@@ -0,0 +1,12 @@
[package]
name = "fparkan-render-gl"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-render = { path = "../../crates/fparkan-render" }
[lints]
workspace = true
+242
View File
@@ -0,0 +1,242 @@
#![forbid(unsafe_code)]
//! OpenGL render adapter boundary stubs behind safe `FParkan` render ports.
use fparkan_render::{
canonical_capture, FrameOutput, RenderBackend, RenderCommandList, RenderError,
};
/// Portable OpenGL profile requested by the game composition root.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum GlProfile {
/// Desktop OpenGL 3.3 Core.
DesktopCore33,
/// OpenGL ES 2.0 portable baseline.
Gles2,
}
/// Shader stage used in diagnostics.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum ShaderStage {
/// Vertex shader.
Vertex,
/// Fragment shader.
Fragment,
}
/// Shader compilation diagnostic surfaced by the adapter.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct ShaderCompileError {
/// Requested GL profile.
pub profile: GlProfile,
/// Shader stage.
pub stage: ShaderStage,
/// Backend compiler log.
pub log: String,
}
impl std::fmt::Display for ShaderCompileError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"{:?} {:?} shader compile failed: {}",
self.profile, self.stage, self.log
)
}
}
impl std::error::Error for ShaderCompileError {}
/// Adapter capabilities compiled into this package.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct GlAdapterCapabilities {
/// Supported profiles in preference order.
pub profiles: Vec<GlProfile>,
/// Whether adapter-owned code is free of `unsafe`.
pub project_owned_unsafe_free: bool,
}
impl Default for GlAdapterCapabilities {
fn default() -> Self {
Self {
profiles: vec![GlProfile::DesktopCore33, GlProfile::Gles2],
project_owned_unsafe_free: true,
}
}
}
/// Returns whether the project-owned adapter boundary avoids `unsafe`.
#[must_use]
pub fn project_owned_layer_unsafe_free() -> bool {
GlAdapterCapabilities::default().project_owned_unsafe_free
}
/// Validates shader source through the adapter diagnostic contract.
///
/// # Errors
///
/// Returns [`ShaderCompileError`] when the source is empty or contains a
/// deterministic synthetic failure marker.
pub fn compile_shader_source(
profile: GlProfile,
stage: ShaderStage,
source: &str,
) -> Result<(), ShaderCompileError> {
if source.trim().is_empty() {
return Err(ShaderCompileError {
profile,
stage,
log: "empty shader source".to_string(),
});
}
if source.contains("#error") {
return Err(ShaderCompileError {
profile,
stage,
log: "synthetic compiler failure marker".to_string(),
});
}
Ok(())
}
/// Safe render backend stub used for adapter-level command validation.
///
/// A concrete OpenGL implementation can be injected behind the same
/// [`RenderBackend`] port once an audited safe GL facade is selected. This type
/// keeps the project-owned adapter API executable without introducing local FFI.
#[derive(Clone, Debug)]
pub struct SafeGlCommandBackend {
profile: GlProfile,
captures: Vec<Vec<u8>>,
}
impl SafeGlCommandBackend {
/// Creates a backend proof for a requested GL profile.
#[must_use]
pub fn new(profile: GlProfile) -> Self {
Self {
profile,
captures: Vec::new(),
}
}
/// Active GL profile.
#[must_use]
pub fn profile(&self) -> GlProfile {
self.profile
}
/// Deterministic command captures produced by executed frames.
#[must_use]
pub fn captures(&self) -> &[Vec<u8>] {
&self.captures
}
}
impl RenderBackend for SafeGlCommandBackend {
fn execute(&mut self, commands: &RenderCommandList) -> Result<FrameOutput, RenderError> {
self.captures.push(canonical_capture(commands)?);
Ok(FrameOutput)
}
}
#[cfg(test)]
mod tests {
use super::*;
use fparkan_render::{
DrawCommand, DrawId, GpuMaterialId, GpuMeshId, IndexRange, RenderCommand, RenderPhase,
};
#[test]
fn adapter_boundary_is_project_owned_unsafe_free() {
assert!(project_owned_layer_unsafe_free());
assert_eq!(GlAdapterCapabilities::default().profiles.len(), 2);
}
#[test]
fn backend_executes_and_captures_commands() -> Result<(), RenderError> {
let mut backend = SafeGlCommandBackend::new(GlProfile::Gles2);
let commands = RenderCommandList {
commands: vec![
RenderCommand::BeginFrame,
RenderCommand::Draw(DrawCommand {
id: DrawId(7),
phase: RenderPhase::Opaque,
object_id: None,
mesh: GpuMeshId(11),
material: GpuMaterialId(13),
transform: [0.0; 16],
range: IndexRange { start: 0, count: 3 },
stable_order: 17,
}),
RenderCommand::EndFrame,
],
};
backend.execute(&commands)?;
assert_eq!(backend.profile(), GlProfile::Gles2);
assert_eq!(backend.captures().len(), 1);
Ok(())
}
#[test]
fn desktop_gl33_triangle_command_capture() -> Result<(), RenderError> {
let mut backend = SafeGlCommandBackend::new(GlProfile::DesktopCore33);
let commands = triangle_commands();
backend.execute(&commands)?;
assert_eq!(backend.profile(), GlProfile::DesktopCore33);
assert_eq!(
backend.captures(),
&[b"B\nD,Opaque,7,11,13,17\nE\n".to_vec()]
);
Ok(())
}
#[test]
fn gles2_triangle_command_capture() -> Result<(), RenderError> {
let mut backend = SafeGlCommandBackend::new(GlProfile::Gles2);
let commands = triangle_commands();
backend.execute(&commands)?;
assert_eq!(backend.profile(), GlProfile::Gles2);
assert_eq!(
backend.captures(),
&[b"B\nD,Opaque,7,11,13,17\nE\n".to_vec()]
);
Ok(())
}
#[test]
fn shader_compile_failure_diagnostic_contains_profile_and_log() {
let err = compile_shader_source(GlProfile::Gles2, ShaderStage::Fragment, "#error")
.expect_err("shader failure");
assert_eq!(err.profile, GlProfile::Gles2);
assert_eq!(err.stage, ShaderStage::Fragment);
assert!(err.log.contains("synthetic compiler failure"));
assert!(err.to_string().contains("Gles2"));
assert!(err.to_string().contains("synthetic compiler failure"));
}
fn triangle_commands() -> RenderCommandList {
RenderCommandList {
commands: vec![
RenderCommand::BeginFrame,
RenderCommand::Draw(DrawCommand {
id: DrawId(7),
phase: RenderPhase::Opaque,
object_id: None,
mesh: GpuMeshId(11),
material: GpuMaterialId(13),
transform: [0.0; 16],
range: IndexRange { start: 0, count: 3 },
stable_order: 17,
}),
RenderCommand::EndFrame,
],
}
}
}
+5
View File
@@ -0,0 +1,5 @@
# ADR-0001: Modular Monolith
Status: accepted
FParkan is implemented as one Cargo workspace with local crates grouped by domain. Binaries and adapters compose domain crates; domain crates do not import platform, windowing, OpenGL, GUI, or application packages.
+5
View File
@@ -0,0 +1,5 @@
# ADR-0002: Behavior Compatibility, Not ABI Compatibility
Status: accepted
The project targets clean-room behavior compatibility for formats, resource lookup, loading order, deterministic runtime behavior, and presentation command semantics. It does not reproduce original DLL boundaries, exports, calling conventions, object layouts, RVAs, or native singleton access patterns.
+5
View File
@@ -0,0 +1,5 @@
# ADR-0003: Raw And Interpreted Data
Status: accepted
Legacy data models keep raw bytes distinct from validated structure and interpreted domain views. Writers preserve raw data unless an explicit editing profile requests canonical rebuilding.
@@ -0,0 +1,5 @@
# ADR-0004: Synthetic And Licensed Tests
Status: accepted
Synthetic tests run everywhere and contain no proprietary data. Licensed corpus tests require an explicit local manifest and fail when requested without configuration. Reports contain metrics and fingerprints, not payload dumps or absolute game roots.
@@ -0,0 +1,5 @@
# ADR-0005: Deterministic Reference Runtime
Status: accepted
Stages 0-5 use a single-threaded deterministic reference profile. Stable ordering, explicit ticks, named random streams, and canonical captures are part of the contract. Wall-clock time, pointer addresses, hash iteration order, and GPU handles are not semantic inputs.
+5
View File
@@ -0,0 +1,5 @@
# ADR-0006: Error Policy
Status: accepted
Missing required resources, malformed bytes, unsupported documented branches, capability mismatches, and budget failures are structured errors. Runtime code must not silently skip mandatory objects or convert corrupted data into empty success values.
+26
View File
@@ -0,0 +1,26 @@
# ADR-0007: Safe SDL/OpenGL Boundary
Status: provisional
Workspace-owned code forbids `unsafe`. SDL/OpenGL adapters must use maintained
external crates behind a safe project API; local Objective-C/CGL/SDL/OpenGL FFI
inside FParkan is not an acceptable implementation strategy.
The current adapter crates are safe boundary stubs. They compile the intended
ports and deterministic command contracts, but they do not create SDL windows,
GL contexts, GPU resources, shaders, draw calls, swapchains, or presents. They
must not be treated as backend readiness evidence.
To close the macOS backend requirement, choose and vendor/lock a maintained
safe facade stack, then implement:
- SDL event source, window creation, GL context lifecycle, drawable size and
present;
- GL shader compile/link, buffer/texture upload, render state, draw calls and
diagnostics;
- game/viewer composition roots using those adapters;
- hidden-window/offscreen macOS smoke tests and licensed local model/terrain
frame captures.
Until those are implemented, Desktop GL evidence may document external probes
only; it does not satisfy the permanent adapter requirement.
+18
View File
@@ -0,0 +1,18 @@
[package]
name = "fparkan-cli"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-corpus = { path = "../../crates/fparkan-corpus" }
fparkan-nres = { path = "../../crates/fparkan-nres" }
fparkan-prototype = { path = "../../crates/fparkan-prototype" }
fparkan-resource = { path = "../../crates/fparkan-resource" }
fparkan-rsli = { path = "../../crates/fparkan-rsli" }
fparkan-runtime = { path = "../../crates/fparkan-runtime" }
fparkan-vfs = { path = "../../crates/fparkan-vfs" }
[lints]
workspace = true
+349
View File
@@ -0,0 +1,349 @@
#![forbid(unsafe_code)]
#![allow(clippy::print_stderr, clippy::print_stdout)]
//! `FParkan` command-line tools.
use fparkan_corpus::{discover, render_report_json, report, DiscoverOptions};
use fparkan_prototype::{
build_prototype_graph_report, extend_graph_report_with_visual_dependencies,
};
use fparkan_resource::{resource_name, CachedResourceRepository};
use fparkan_runtime::{
create, load_mission, EngineConfig, EngineMode, EngineServices, MissionRequest,
};
use fparkan_vfs::DirectoryVfs;
use std::fmt::Write;
use std::path::PathBuf;
use std::sync::Arc;
fn main() {
let args: Vec<String> = std::env::args().skip(1).collect();
let result = run(&args);
let code = exit_code(&result);
if let Err(err) = result {
eprintln!("{err}");
}
std::process::exit(code);
}
fn run(args: &[String]) -> Result<(), String> {
match args {
[domain, command, rest @ ..] if domain == "corpus" && command == "discover" => {
let rest = strip_format_json(rest)?;
let root = parse_root(&rest)?;
let manifest =
discover(&root, DiscoverOptions::default()).map_err(|e| e.to_string())?;
let report = report(&root, &manifest).map_err(|e| e.to_string())?;
println!("{}", render_report_json(&report));
Ok(())
}
[domain, command, rest @ ..] if domain == "corpus" && command == "validate" => {
let rest = strip_format_json(rest)?;
let root = parse_root(&rest)?;
let manifest =
discover(&root, DiscoverOptions::default()).map_err(|e| e.to_string())?;
let report = report(&root, &manifest).map_err(|e| e.to_string())?;
if report.casefold_collisions > 0 {
return Err("casefold collisions found".to_string());
}
if report.failures > 0 {
return Err(format!("corpus report found {} failures", report.failures));
}
println!("{}", render_report_json(&report));
Ok(())
}
[domain, command, rest @ ..] if domain == "archive" && command == "inspect" => {
let rest = strip_format_json(rest)?;
inspect_archive(&rest)
}
[domain, command, rest @ ..] if domain == "prototype" && command == "inspect" => {
let rest = strip_format_json(rest)?;
inspect_prototype(&rest)
}
[domain, command, rest @ ..] if domain == "mission" && command == "graph" => {
let rest = strip_format_json(rest)?;
graph_mission(&rest)
}
_ => Err(usage()),
}
}
fn exit_code(result: &Result<(), String>) -> i32 {
if result.is_ok() {
0
} else {
2
}
}
fn strip_format_json(args: &[String]) -> Result<Vec<String>, String> {
let mut stripped = Vec::with_capacity(args.len());
let mut iter = args.iter();
while let Some(arg) = iter.next() {
if arg == "--format" {
let value = iter
.next()
.ok_or_else(|| "--format requires a value".to_string())?;
if value != "json" {
return Err(format!("unsupported output format: {value}"));
}
continue;
}
stripped.push(arg.clone());
}
Ok(stripped)
}
fn parse_root(args: &[String]) -> Result<PathBuf, String> {
let mut iter = args.iter();
while let Some(arg) = iter.next() {
if arg == "--root" {
return iter
.next()
.map(PathBuf::from)
.ok_or_else(|| "--root requires a path".to_string());
}
}
Err("missing --root".to_string())
}
fn parse_root_alias(args: &[String]) -> Result<PathBuf, String> {
parse_option(args, &["--root", "--game-root"])
.map(PathBuf::from)
.ok_or_else(|| "missing --root".to_string())
}
fn parse_required(args: &[String], names: &[&str], label: &str) -> Result<String, String> {
parse_option(args, names).ok_or_else(|| format!("missing {label}"))
}
fn parse_option(args: &[String], names: &[&str]) -> Option<String> {
let mut iter = args.iter();
while let Some(arg) = iter.next() {
if names.iter().any(|name| arg == name) {
return iter.next().cloned();
}
}
None
}
fn inspect_prototype(args: &[String]) -> Result<(), String> {
let root = parse_root_alias(args)?;
let key = parse_required(args, &["--key"], "--key")?;
let vfs = Arc::new(DirectoryVfs::new(root));
let repository = CachedResourceRepository::new(vfs.clone());
let roots = [resource_name(key.as_bytes())];
let (graph, resolved, mut report) =
build_prototype_graph_report(&repository, vfs.as_ref(), &roots);
extend_graph_report_with_visual_dependencies(&repository, &mut report, &resolved);
println!("{}", prototype_inspect_json(&key, &graph, &report));
Ok(())
}
fn prototype_inspect_json(
key: &str,
graph: &fparkan_prototype::PrototypeGraph,
report: &fparkan_prototype::PrototypeGraphReport,
) -> String {
format!(
"{{\"schema_version\":\"fparkan-prototype-inspect-v1\",\"key\":{},\"roots\":{},\"prototype_requests\":{},\"resolved\":{},\"unit_references\":{},\"unit_components\":{},\"direct_references\":{},\"wear\":{},\"materials\":{},\"textures\":{},\"lightmaps\":{},\"failures\":{}}}",
json_string(key),
report.root_count,
graph.prototype_requests.len(),
report.resolved_count,
report.unit_reference_count,
report.unit_component_count,
report.direct_reference_count,
report.wear_resolved_count,
report.material_resolved_count,
report.texture_resolved_count,
report.lightmap_resolved_count,
report.failures.len()
)
}
fn graph_mission(args: &[String]) -> Result<(), String> {
let root = parse_root_alias(args)?;
let mission = parse_required(args, &["--mission"], "--mission")?;
let services = EngineServices::new(Arc::new(DirectoryVfs::new(root)));
let mut engine = create(
EngineConfig {
mode: EngineMode::Headless,
},
services,
)
.map_err(|err| err.to_string())?;
let loaded = load_mission(
&mut engine,
MissionRequest {
key: mission.clone(),
},
)
.map_err(|err| err.to_string())?;
println!(
"{{\"schema_version\":\"fparkan-mission-graph-v1\",\"mission\":{},\"objects\":{},\"paths\":{},\"clans\":{},\"extras\":{},\"roots\":{},\"direct_references\":{},\"unit_references\":{},\"unit_components\":{},\"prototype_requests\":{},\"wear\":{},\"materials\":{},\"textures\":{},\"lightmaps\":{},\"failures\":{}}}",
json_string(&mission),
loaded.object_count,
loaded.path_count,
loaded.clan_count,
loaded.extra_count,
loaded.graph_root_count,
loaded.graph_direct_reference_count,
loaded.graph_unit_reference_count,
loaded.graph_unit_component_count,
loaded.graph_resolved_count,
loaded.graph_wear_resolved_count,
loaded.graph_material_resolved_count,
loaded.graph_texture_resolved_count,
loaded.graph_lightmap_resolved_count,
loaded.graph_failure_count
);
Ok(())
}
fn inspect_archive(args: &[String]) -> Result<(), String> {
let path = parse_archive_path(args)?;
let bytes = std::fs::read(&path).map_err(|err| format!("{}: {err}", path.display()))?;
if bytes.starts_with(b"NRes") {
let document = fparkan_nres::decode(
Arc::from(bytes.into_boxed_slice()),
fparkan_nres::ReadProfile::Compatible,
)
.map_err(|err| err.to_string())?;
println!(
"{}",
archive_inspect_json(
&path.display().to_string(),
"NRes",
document.entries().len(),
Some(document.lookup_order_valid()),
)
);
return Ok(());
}
if bytes.get(0..4) == Some(b"NL\0\x01") {
let document = fparkan_rsli::decode(
Arc::from(bytes.into_boxed_slice()),
fparkan_rsli::ReadProfile::Compatible,
)
.map_err(|err| err.to_string())?;
println!(
"{}",
archive_inspect_json(
&path.display().to_string(),
"RsLi",
document.entries().len(),
None
)
);
return Ok(());
}
Err(format!("{}: unsupported archive magic", path.display()))
}
fn archive_inspect_json(
path: &str,
kind: &str,
entries: usize,
lookup_order_valid: Option<bool>,
) -> String {
let mut out = format!(
"{{\"schema_version\":\"fparkan-archive-inspect-v1\",\"path\":{},\"kind\":{},\"entries\":{}",
json_string(path),
json_string(kind),
entries
);
if let Some(valid) = lookup_order_valid {
let _ = write!(out, ",\"lookup_order_valid\":{valid}");
}
out.push('}');
out
}
fn parse_archive_path(args: &[String]) -> Result<PathBuf, String> {
match args {
[path] => Ok(PathBuf::from(path)),
[flag, path] if flag == "--file" => Ok(PathBuf::from(path)),
_ => Err("archive inspect requires <file> or --file <file>".to_string()),
}
}
fn json_string(value: &str) -> String {
let mut out = String::with_capacity(value.len() + 2);
out.push('"');
for ch in value.chars() {
match ch {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if c.is_control() => {
let _ = write!(out, "\\u{:04x}", u32::from(c));
}
c => out.push(c),
}
}
out.push('"');
out
}
fn usage() -> String {
"usage: fparkan corpus discover|validate --root <path> [--format json] | archive inspect <file> [--format json] | prototype inspect --root <path> --key <key> [--format json] | mission graph --root <path> --mission <path> [--format json]".to_string()
}
#[cfg(test)]
mod tests {
use super::*;
fn strings(values: &[&str]) -> Vec<String> {
values.iter().map(|value| (*value).to_string()).collect()
}
#[test]
fn stable_exit_codes_are_mapped() {
assert_eq!(exit_code(&Ok(())), 0);
assert_eq!(exit_code(&Err("failure".to_string())), 2);
}
#[test]
fn accepts_json_format_option() {
assert_eq!(
strip_format_json(&strings(&["--root", "testdata", "--format", "json"])),
Ok(strings(&["--root", "testdata"]))
);
assert_eq!(
strip_format_json(&strings(&["--format", "text"])),
Err("unsupported output format: text".to_string())
);
}
#[test]
fn archive_json_has_schema_version() {
let json = archive_inspect_json("archive.lib", "NRes", 3, Some(true));
assert!(json.contains("\"schema_version\":\"fparkan-archive-inspect-v1\""));
assert!(json.contains("\"kind\":\"NRes\""));
assert!(json.contains("\"lookup_order_valid\":true"));
}
#[test]
fn prototype_graph_json_has_canonical_field_order() {
let mut graph = fparkan_prototype::PrototypeGraph::default();
graph
.prototype_requests
.push(fparkan_prototype::PrototypeKey(resource_name(b"root")));
let report = fparkan_prototype::PrototypeGraphReport {
root_count: 1,
direct_reference_count: 1,
resolved_count: 1,
..fparkan_prototype::PrototypeGraphReport::default()
};
let json = prototype_inspect_json("root", &graph, &report);
assert_eq!(
json,
"{\"schema_version\":\"fparkan-prototype-inspect-v1\",\"key\":\"root\",\"roots\":1,\"prototype_requests\":1,\"resolved\":1,\"unit_references\":0,\"unit_components\":0,\"direct_references\":1,\"wear\":0,\"materials\":0,\"textures\":0,\"lightmaps\":0,\"failures\":0}"
);
}
}
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "fparkan-game"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-render = { path = "../../crates/fparkan-render" }
fparkan-runtime = { path = "../../crates/fparkan-runtime" }
fparkan-vfs = { path = "../../crates/fparkan-vfs" }
fparkan-world = { path = "../../crates/fparkan-world" }
[lints]
workspace = true
+332
View File
@@ -0,0 +1,332 @@
#![forbid(unsafe_code)]
#![allow(clippy::print_stderr, clippy::print_stdout)]
//! `FParkan` rendered game composition root.
use fparkan_render::{
DrawCommand, DrawId, GpuMaterialId, GpuMeshId, IndexRange, RecordingBackend, RenderBackend,
RenderCommand, RenderCommandList, RenderPhase,
};
use fparkan_runtime::{
create, frame, load_mission, EngineConfig, EngineMode, EngineServices, MissionRequest,
};
use fparkan_vfs::DirectoryVfs;
use fparkan_world::WorldSnapshot;
use std::path::PathBuf;
use std::sync::Arc;
fn main() {
let raw_args = std::env::args().skip(1).collect::<Vec<_>>();
let code = match run(&raw_args) {
Ok(output) => {
println!("{output}");
0
}
Err(err) => {
eprintln!("{err}");
2
}
};
std::process::exit(code);
}
fn run(args: &[String]) -> Result<String, String> {
let args = Args::parse(args)?;
let services = EngineServices::new(Arc::new(DirectoryVfs::new(&args.root)));
let mut engine = create(
EngineConfig {
mode: EngineMode::Rendered,
},
services,
)
.map_err(|err| err.to_string())?;
let loaded = load_mission(
&mut engine,
MissionRequest {
key: args.mission.clone(),
},
)
.map_err(|err| err.to_string())?;
let mut backend = RecordingBackend::default();
let mut last_draw_count = 0usize;
let mut last_tick = 0u64;
let mut last_hash = [0u8; 32];
for _ in 0..args.frames {
let result = frame(&mut engine).map_err(|err| err.to_string())?;
last_tick = result.snapshot.tick.0;
last_hash = result.snapshot.hash.0;
let commands = render_snapshot_commands(&result.snapshot);
last_draw_count = commands
.commands
.iter()
.filter(|command| matches!(command, RenderCommand::Draw(_)))
.count();
backend
.execute(&commands)
.map_err(|err| format!("render backend: {err}"))?;
}
Ok(format!(
"{{\"mission\":{},\"objects\":{},\"frames\":{},\"tick\":{},\"draws\":{},\"captures\":{},\"last_capture_bytes\":{},\"hash\":{}}}",
json_string(&args.mission),
loaded.object_count,
args.frames,
last_tick,
last_draw_count,
backend.captures().len(),
backend.last_capture().map_or(0, <[u8]>::len),
json_hash(&last_hash)
))
}
fn render_snapshot_commands(snapshot: &WorldSnapshot) -> RenderCommandList {
let mut commands = Vec::with_capacity(snapshot.objects.len() + 2);
commands.push(RenderCommand::BeginFrame);
for (index, handle) in snapshot.objects.iter().enumerate() {
let stable_order = u64::from(handle.slot);
let draw_id = snapshot
.tick
.0
.wrapping_mul(1_000_003)
.wrapping_add(stable_order);
commands.push(RenderCommand::Draw(DrawCommand {
id: DrawId(draw_id),
phase: RenderPhase::Opaque,
object_id: None,
mesh: GpuMeshId(u64::from(handle.slot) + 1),
material: GpuMaterialId(1),
transform: identity_transform(index_to_f32(index)),
range: IndexRange { start: 0, count: 3 },
stable_order,
}));
}
commands.push(RenderCommand::EndFrame);
RenderCommandList { commands }
}
fn identity_transform(x: f32) -> [f32; 16] {
[
1.0, 0.0, 0.0, x, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0,
]
}
fn index_to_f32(index: usize) -> f32 {
u16::try_from(index).map_or(f32::from(u16::MAX), f32::from)
}
#[derive(Clone, Debug, Eq, PartialEq)]
struct Args {
root: PathBuf,
mission: String,
frames: u64,
}
impl Args {
fn parse(args: &[String]) -> Result<Self, String> {
let mut root = None;
let mut mission = None;
let mut frames = 1;
let mut iter = args.iter();
while let Some(arg) = iter.next() {
match arg.as_str() {
"--root" => {
root = Some(
iter.next()
.map(PathBuf::from)
.ok_or_else(|| "--root requires a path".to_string())?,
);
}
"--mission" => {
mission = Some(
iter.next()
.cloned()
.ok_or_else(|| "--mission requires a path".to_string())?,
);
}
"--frames" => {
frames = iter
.next()
.ok_or_else(|| "--frames requires a value".to_string())?
.parse()
.map_err(|_| "--frames must be an integer".to_string())?;
}
_ => return Err(usage()),
}
}
let root = root.ok_or_else(|| "missing --root".to_string())?;
let mission = mission.ok_or_else(|| "missing --mission".to_string())?;
if frames == 0 {
return Err("--frames must be greater than zero".to_string());
}
Ok(Self {
root,
mission,
frames,
})
}
}
fn json_string(value: &str) -> String {
let mut out = String::with_capacity(value.len() + 2);
out.push('"');
for ch in value.chars() {
match ch {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if c.is_control() => {
use std::fmt::Write as _;
let _ = write!(out, "\\u{:04x}", u32::from(c));
}
c => out.push(c),
}
}
out.push('"');
out
}
fn json_hash(hash: &[u8; 32]) -> String {
let mut out = String::from("\"");
for byte in hash {
use std::fmt::Write as _;
let _ = write!(out, "{byte:02x}");
}
out.push('"');
out
}
fn usage() -> String {
"usage: fparkan-game --root <path> --mission <path> [--frames <n>]".to_string()
}
#[cfg(test)]
mod tests {
use super::*;
use fparkan_world::{ObjectHandle, StateHash, Tick};
use std::path::Path;
fn strings(values: &[&str]) -> Vec<String> {
values.iter().map(|value| (*value).to_string()).collect()
}
#[test]
fn parses_required_args() {
assert_eq!(
Args::parse(&strings(&[
"--root",
"testdata/IS",
"--mission",
"MISSIONS/Autodemo.00/data.tma",
"--frames",
"3",
])),
Ok(Args {
root: PathBuf::from("testdata/IS"),
mission: "MISSIONS/Autodemo.00/data.tma".to_string(),
frames: 3,
})
);
}
#[test]
fn render_commands_follow_snapshot_order() -> Result<(), String> {
let snapshot = WorldSnapshot {
tick: Tick(7),
objects: vec![
ObjectHandle {
generation: 1,
slot: 2,
},
ObjectHandle {
generation: 1,
slot: 5,
},
],
events: Vec::new(),
hash: StateHash([0; 32]),
};
let commands = render_snapshot_commands(&snapshot);
assert_eq!(commands.commands.len(), 4);
assert!(matches!(commands.commands[0], RenderCommand::BeginFrame));
assert!(matches!(commands.commands[3], RenderCommand::EndFrame));
let RenderCommand::Draw(first) = &commands.commands[1] else {
return Err("expected draw".to_string());
};
assert_eq!(first.mesh, GpuMeshId(3));
assert_eq!(first.stable_order, 2);
Ok(())
}
#[test]
#[ignore = "requires licensed corpus"]
fn selected_is_and_is2_missions_produce_approved_render_captures() {
for case in [
RenderCase {
root: "IS",
mission: "MISSIONS/CAMPAIGN/CAMPAIGN.00/Mission.01/data.tma",
expected: "{\"mission\":\"MISSIONS/CAMPAIGN/CAMPAIGN.00/Mission.01/data.tma\",\"objects\":33,\"frames\":1,\"tick\":1,\"draws\":33,\"captures\":1,\"last_capture_bytes\":810,\"hash\":\"ca17cc76e55c45e83c1c9c1c088e84bf1a698be91a7730943210fe27596af841\"}",
},
RenderCase {
root: "IS2",
mission: "MISSIONS/Campaign/CAMPAIGN.00/Mission.02/data.tma",
expected: "{\"mission\":\"MISSIONS/Campaign/CAMPAIGN.00/Mission.02/data.tma\",\"objects\":10,\"frames\":1,\"tick\":1,\"draws\":10,\"captures\":1,\"last_capture_bytes\":235,\"hash\":\"5d720b3ab690076a398a79a404850bbeaee2e33811b5bb570ec8a96d4a7a2fc4\"}",
},
] {
assert_eq!(
run(&render_args(&licensed_root(case.root), case.mission)),
Ok(case.expected.to_string())
);
}
}
#[test]
fn json_hash_is_hex() {
let mut hash = [0; 32];
hash[0] = 0xab;
hash[31] = 0xcd;
assert_eq!(
json_hash(&hash),
"\"ab000000000000000000000000000000000000000000000000000000000000cd\""
);
}
#[derive(Clone, Copy)]
struct RenderCase {
root: &'static str,
mission: &'static str,
expected: &'static str,
}
fn render_args(root: &Path, mission: &str) -> Vec<String> {
vec![
"--root".to_string(),
root.to_str().expect("utf8 root").to_string(),
"--mission".to_string(),
mission.to_string(),
"--frames".to_string(),
"1".to_string(),
]
}
fn licensed_root(name: &str) -> PathBuf {
let variable = match name {
"IS" => "FPARKAN_CORPUS_PART1_ROOT",
"IS2" => "FPARKAN_CORPUS_PART2_ROOT",
_ => panic!("unknown licensed corpus part: {name}"),
};
let root = std::env::var_os(variable)
.map(PathBuf::from)
.unwrap_or_else(|| panic!("{variable} is required for licensed corpus tests"));
assert!(
root.is_dir(),
"licensed corpus root is missing: {}",
root.display()
);
root
}
}
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "fparkan-headless"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-runtime = { path = "../../crates/fparkan-runtime" }
fparkan-vfs = { path = "../../crates/fparkan-vfs" }
fparkan-world = { path = "../../crates/fparkan-world" }
[lints]
workspace = true
+114
View File
@@ -0,0 +1,114 @@
#![forbid(unsafe_code)]
#![allow(clippy::print_stderr, clippy::print_stdout)]
//! `FParkan` headless runtime entrypoint.
use fparkan_runtime::{
create, load_mission, step_headless, EngineConfig, EngineMode, EngineServices, MissionRequest,
};
use fparkan_vfs::DirectoryVfs;
use fparkan_world::InputSnapshot;
use std::path::PathBuf;
use std::sync::Arc;
fn main() {
if let Err(err) = run() {
eprintln!("{err}");
std::process::exit(2);
}
}
fn run() -> Result<(), String> {
let raw_args: Vec<String> = std::env::args().skip(1).collect();
let args = Args::parse(&raw_args)?;
let services = if let Some(root) = &args.root {
EngineServices::new(Arc::new(DirectoryVfs::new(root)))
} else {
EngineServices::default()
};
let mut engine = create(
EngineConfig {
mode: EngineMode::Headless,
},
services,
)
.map_err(|err| format!("{err}"))?;
if let Some(mission) = args.mission {
let loaded = load_mission(&mut engine, MissionRequest { key: mission })
.map_err(|err| format!("{err}"))?;
println!(
"mission objects={} areals={} surfaces={} graph_roots={} components={} wear={} material_slots={} textures={} lightmaps={} graph_failures={}",
loaded.object_count,
loaded.areal_count,
loaded.surface_count,
loaded.graph_root_count,
loaded.graph_unit_component_count,
loaded.graph_wear_resolved_count,
loaded.graph_material_resolved_count,
loaded.graph_texture_resolved_count,
loaded.graph_lightmap_resolved_count,
loaded.graph_failure_count
);
}
let mut last = None;
for _ in 0..args.ticks {
last = Some(step_headless(&mut engine, InputSnapshot).map_err(|err| format!("{err}"))?);
}
if let Some(frame) = last {
println!(
"tick={} hash={:02x?}",
frame.snapshot.tick.0, frame.snapshot.hash.0
);
}
Ok(())
}
struct Args {
root: Option<PathBuf>,
mission: Option<String>,
ticks: u64,
}
impl Args {
fn parse(args: &[String]) -> Result<Self, String> {
let mut parsed = Self {
root: None,
mission: None,
ticks: 1,
};
let mut iter = args.iter();
while let Some(arg) = iter.next() {
match arg.as_str() {
"--root" => {
parsed.root = Some(
iter.next()
.map(PathBuf::from)
.ok_or_else(|| "--root requires a path".to_string())?,
);
}
"--mission" => {
parsed.mission = Some(
iter.next()
.cloned()
.ok_or_else(|| "--mission requires a path".to_string())?,
);
}
"--ticks" => {
parsed.ticks = iter
.next()
.ok_or_else(|| "--ticks requires a value".to_string())?
.parse()
.map_err(|_| "--ticks must be an integer".to_string())?;
}
_ => return Err(usage()),
}
}
if parsed.mission.is_some() && parsed.root.is_none() {
return Err("--mission requires --root".to_string());
}
Ok(parsed)
}
}
fn usage() -> String {
"usage: fparkan-headless [--root <path> --mission <path>] [--ticks <n>]".to_string()
}
+19
View File
@@ -0,0 +1,19 @@
[package]
name = "fparkan-viewer"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-msh = { path = "../../crates/fparkan-msh" }
fparkan-nres = { path = "../../crates/fparkan-nres" }
fparkan-resource = { path = "../../crates/fparkan-resource" }
fparkan-render = { path = "../../crates/fparkan-render" }
fparkan-rsli = { path = "../../crates/fparkan-rsli" }
fparkan-terrain-format = { path = "../../crates/fparkan-terrain-format" }
fparkan-texm = { path = "../../crates/fparkan-texm" }
fparkan-vfs = { path = "../../crates/fparkan-vfs" }
[lints]
workspace = true
+353
View File
@@ -0,0 +1,353 @@
#![forbid(unsafe_code)]
#![allow(clippy::print_stderr, clippy::print_stdout)]
//! `FParkan` asset viewer composition root.
use fparkan_msh::{decode_msh, validate_msh};
use fparkan_nres::{decode as decode_nres, ReadProfile as NresReadProfile};
use fparkan_render::{
build_commands, CameraSnapshot, DrawId, GpuMaterialId, GpuMeshId, IndexRange, RenderPhase,
RenderProfile, RenderSnapshot, RenderSnapshotDraw,
};
use fparkan_resource::{archive_path, resource_name, CachedResourceRepository, ResourceRepository};
use fparkan_terrain_format::{decode_land_map, decode_land_msh};
use fparkan_texm::decode_texm;
use fparkan_vfs::DirectoryVfs;
use std::fmt::Write;
use std::path::PathBuf;
use std::sync::Arc;
fn main() {
let args = std::env::args().skip(1).collect::<Vec<_>>();
let code = match run(&args) {
Ok(json) => {
println!("{json}");
0
}
Err(err) => {
eprintln!("{err}");
2
}
};
std::process::exit(code);
}
fn run(args: &[String]) -> Result<String, String> {
match args {
[domain, rest @ ..] if domain == "archive" => inspect_archive(rest),
[domain, rest @ ..] if domain == "model" => inspect_model(rest),
[domain, rest @ ..] if domain == "texture" => inspect_texture(rest),
[domain, rest @ ..] if domain == "map" => inspect_map(rest),
_ => Err(usage()),
}
}
fn inspect_archive(args: &[String]) -> Result<String, String> {
let file = parse_file(args)?;
let limit = parse_limit(args)?;
let bytes = std::fs::read(&file).map_err(|err| format!("{}: {err}", file.display()))?;
if bytes.starts_with(b"NRes") {
let document = decode_nres(
Arc::from(bytes.into_boxed_slice()),
NresReadProfile::Compatible,
)
.map_err(|err| err.to_string())?;
let sample = render_nres_entries(&document, limit);
return Ok(format!(
"{{\"kind\":\"NRes\",\"path\":{},\"entries\":{},\"lookup_order_valid\":{},\"sample\":[{}]}}",
json_string(&file.display().to_string()),
document.entries().len(),
document.lookup_order_valid(),
sample
));
}
if bytes.get(0..4) == Some(b"NL\0\x01") {
let document = fparkan_rsli::decode(
Arc::from(bytes.into_boxed_slice()),
fparkan_rsli::ReadProfile::Compatible,
)
.map_err(|err| err.to_string())?;
return Ok(format!(
"{{\"kind\":\"RsLi\",\"path\":{},\"entries\":{}}}",
json_string(&file.display().to_string()),
document.entries().len()
));
}
Err(format!("{}: unsupported archive magic", file.display()))
}
fn inspect_model(args: &[String]) -> Result<String, String> {
if let Some(fixture) = parse_option(args, &["--fixture"]) {
return ViewerModelService::inspect_synthetic_model(&fixture);
}
let query = parse_resource_query(args)?;
let bytes = read_resource(&query)?;
let nested = decode_nres(bytes, NresReadProfile::Compatible).map_err(|err| err.to_string())?;
let document = decode_msh(&nested).map_err(|err| err.to_string())?;
let model = validate_msh(&document).map_err(|err| err.to_string())?;
Ok(format!(
"{{\"kind\":\"model\",\"archive\":{},\"name\":{},\"streams\":{},\"nodes\":{},\"slots\":{},\"positions\":{},\"indices\":{},\"batches\":{}}}",
json_string(&query.archive),
json_string(&query.name),
document.streams().len(),
model.node_count,
model.slots.len(),
model.positions.len(),
model.indices.len(),
model.batches.len()
))
}
#[derive(Clone, Debug)]
struct ViewerModelService;
impl ViewerModelService {
fn inspect_synthetic_model(fixture: &str) -> Result<String, String> {
if fixture != "synthetic/model-basic" {
return Err(format!("unknown model fixture: {fixture}"));
}
let snapshot = RenderSnapshot {
camera: CameraSnapshot::default(),
draws: vec![RenderSnapshotDraw {
id: DrawId(1),
phase: RenderPhase::Opaque,
object_id: None,
mesh: GpuMeshId(1),
material_slots: vec![GpuMaterialId(7)],
material_index: 0,
transform: identity_transform(),
range: IndexRange { start: 0, count: 3 },
stable_order: 0,
}],
};
let commands = build_commands(&snapshot, RenderProfile::default())
.map_err(|err| format!("render command generation: {err}"))?;
let draw_commands = commands
.commands
.iter()
.filter(|command| matches!(command, fparkan_render::RenderCommand::Draw(_)))
.count();
Ok(format!(
"{{\"kind\":\"model\",\"fixture\":{},\"service\":\"synthetic-model\",\"draw_commands\":{draw_commands}}}",
json_string(fixture)
))
}
}
fn inspect_texture(args: &[String]) -> Result<String, String> {
let query = parse_resource_query(args)?;
let document = decode_texm(read_resource(&query)?).map_err(|err| err.to_string())?;
Ok(format!(
"{{\"kind\":\"texture\",\"archive\":{},\"name\":{},\"width\":{},\"height\":{},\"format\":{},\"mips\":{},\"pages\":{}}}",
json_string(&query.archive),
json_string(&query.name),
document.width(),
document.height(),
json_string(&format!("{:?}", document.format())),
document.mip_count(),
document.page_rects().len()
))
}
fn inspect_map(args: &[String]) -> Result<String, String> {
let file = parse_file(args)?;
let kind = parse_option(args, &["--kind"]).ok_or_else(|| "missing --kind".to_string())?;
let bytes = std::fs::read(&file).map_err(|err| format!("{}: {err}", file.display()))?;
let nres = decode_nres(
Arc::from(bytes.into_boxed_slice()),
NresReadProfile::Compatible,
)
.map_err(|err| err.to_string())?;
match kind.as_str() {
"land-msh" => {
let land = decode_land_msh(&nres).map_err(|err| err.to_string())?;
Ok(format!(
"{{\"kind\":\"land-msh\",\"path\":{},\"streams\":{},\"positions\":{},\"faces\":{},\"slots\":{}}}",
json_string(&file.display().to_string()),
land.streams.len(),
land.positions.len(),
land.faces.len(),
land.slots.slots_raw.len()
))
}
"land-map" => {
let land = decode_land_map(&nres).map_err(|err| err.to_string())?;
Ok(format!(
"{{\"kind\":\"land-map\",\"path\":{},\"areals\":{},\"declared_areals\":{},\"grid_width\":{},\"grid_height\":{}}}",
json_string(&file.display().to_string()),
land.areals.len(),
land.areal_count,
land.grid.cells_x,
land.grid.cells_y
))
}
_ => Err(format!("unknown map kind: {kind}")),
}
}
struct ResourceQuery {
root: PathBuf,
archive: String,
name: String,
}
fn parse_resource_query(args: &[String]) -> Result<ResourceQuery, String> {
Ok(ResourceQuery {
root: parse_path_option(args, &["--root", "--game-root"], "--root")?,
archive: parse_option(args, &["--archive"])
.ok_or_else(|| "missing --archive".to_string())?,
name: parse_option(args, &["--name"]).ok_or_else(|| "missing --name".to_string())?,
})
}
fn read_resource(query: &ResourceQuery) -> Result<Arc<[u8]>, String> {
let repository = CachedResourceRepository::new(Arc::new(DirectoryVfs::new(&query.root)));
let archive = repository
.open_archive(&archive_path(query.archive.as_bytes()).map_err(|err| err.to_string())?)
.map_err(|err| err.to_string())?;
let entry = repository
.find(archive, &resource_name(query.name.as_bytes()))
.map_err(|err| err.to_string())?
.ok_or_else(|| format!("resource not found: {}/{}", query.archive, query.name))?;
let bytes = repository.read(entry).map_err(|err| err.to_string())?;
Ok(Arc::from(bytes.into_owned()))
}
fn parse_file(args: &[String]) -> Result<PathBuf, String> {
parse_path_option(args, &["--file"], "--file")
}
fn parse_limit(args: &[String]) -> Result<usize, String> {
parse_option(args, &["--limit"])
.map(|value| {
value
.parse::<usize>()
.map_err(|_| format!("invalid --limit: {value}"))
})
.transpose()
.map(|value| value.unwrap_or(0))
}
fn render_nres_entries(document: &fparkan_nres::NresDocument, limit: usize) -> String {
let mut out = String::new();
for (index, entry) in document.entries().iter().take(limit).enumerate() {
if index > 0 {
out.push(',');
}
let name = String::from_utf8_lossy(entry.name_bytes());
let _ = write!(
out,
"{{\"name\":{},\"type\":{},\"size\":{}}}",
json_string(&name),
entry.meta().type_id,
entry.meta().data_size
);
}
out
}
fn parse_path_option(args: &[String], names: &[&str], label: &str) -> Result<PathBuf, String> {
parse_option(args, names)
.map(PathBuf::from)
.ok_or_else(|| format!("missing {label}"))
}
fn parse_option(args: &[String], names: &[&str]) -> Option<String> {
let mut iter = args.iter();
while let Some(arg) = iter.next() {
if names.iter().any(|name| arg == name) {
return iter.next().cloned();
}
}
None
}
fn json_string(value: &str) -> String {
let mut out = String::with_capacity(value.len() + 2);
out.push('"');
for ch in value.chars() {
match ch {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if c.is_control() => {
let _ = write!(out, "\\u{:04x}", u32::from(c));
}
c => out.push(c),
}
}
out.push('"');
out
}
fn identity_transform() -> [f32; 16] {
[
1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0,
]
}
fn usage() -> String {
"usage: fparkan-viewer archive --file <archive> [--limit N] | model --root <game-root> --archive <archive> --name <msh> | model --fixture synthetic/model-basic | texture --root <game-root> --archive <archive> --name <texm> | map --file <Land.msh|Land.map> --kind land-msh|land-map".to_string()
}
#[cfg(test)]
mod tests {
use super::*;
fn strings(values: &[&str]) -> Vec<String> {
values.iter().map(|value| (*value).to_string()).collect()
}
#[test]
fn parses_resource_query() -> Result<(), String> {
let query = parse_resource_query(&strings(&[
"--root",
"testdata/IS",
"--archive",
"textures.lib",
"--name",
"grass.tex",
]))?;
assert_eq!(query.root, PathBuf::from("testdata/IS"));
assert_eq!(query.archive, "textures.lib");
assert_eq!(query.name, "grass.tex");
Ok(())
}
#[test]
fn json_string_escapes_controls() {
assert_eq!(json_string("a\"b\\c\n"), "\"a\\\"b\\\\c\\n\"");
}
#[test]
fn usage_rejects_empty_args() {
assert_eq!(run(&[]), Err(usage()));
}
#[test]
fn parses_limit() {
assert_eq!(parse_limit(&strings(&["--limit", "2"])), Ok(2));
assert_eq!(parse_limit(&[]), Ok(0));
assert_eq!(
parse_limit(&strings(&["--limit", "x"])),
Err("invalid --limit: x".to_string())
);
}
#[test]
fn model_fixture_uses_viewer_service_and_render_commands() -> Result<(), String> {
assert_eq!(
run(&strings(&["model", "--fixture", "synthetic/model-basic"]))?,
"{\"kind\":\"model\",\"fixture\":\"synthetic/model-basic\",\"service\":\"synthetic-model\",\"draw_commands\":1}"
);
Ok(())
}
}
-6
View File
@@ -1,6 +0,0 @@
[package]
name = "common"
version = "0.1.0"
edition = "2021"
[dependencies]
-44
View File
@@ -1,44 +0,0 @@
use std::io;
/// Resource payload that can be either borrowed from mapped bytes or owned.
#[derive(Clone, Debug)]
pub enum ResourceData<'a> {
Borrowed(&'a [u8]),
Owned(Vec<u8>),
}
impl<'a> ResourceData<'a> {
pub fn as_slice(&self) -> &[u8] {
match self {
Self::Borrowed(slice) => slice,
Self::Owned(buf) => buf.as_slice(),
}
}
pub fn into_owned(self) -> Vec<u8> {
match self {
Self::Borrowed(slice) => slice.to_vec(),
Self::Owned(buf) => buf,
}
}
}
impl AsRef<[u8]> for ResourceData<'_> {
fn as_ref(&self) -> &[u8] {
self.as_slice()
}
}
/// Output sink used by `read_into`/`load_into` APIs.
pub trait OutputBuffer {
/// Writes the full payload to the sink, replacing any previous content.
fn write_exact(&mut self, data: &[u8]) -> io::Result<()>;
}
impl OutputBuffer for Vec<u8> {
fn write_exact(&mut self, data: &[u8]) -> io::Result<()> {
self.clear();
self.extend_from_slice(data);
Ok(())
}
}
+11
View File
@@ -0,0 +1,11 @@
[package]
name = "fparkan-animation"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+21
View File
@@ -0,0 +1,21 @@
[package]
name = "fparkan-assets"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-material = { path = "../fparkan-material" }
fparkan-msh = { path = "../fparkan-msh" }
fparkan-nres = { path = "../fparkan-nres" }
fparkan-path = { path = "../fparkan-path" }
fparkan-prototype = { path = "../fparkan-prototype" }
fparkan-resource = { path = "../fparkan-resource" }
fparkan-texm = { path = "../fparkan-texm" }
[dev-dependencies]
fparkan-vfs = { path = "../fparkan-vfs" }
[lints]
workspace = true
+622
View File
@@ -0,0 +1,622 @@
#![forbid(unsafe_code)]
//! Asset manager ports and transactional preparation models.
use fparkan_material::{decode_wear, resolve_material, WEAR_KIND};
use fparkan_msh::{decode_msh, validate_msh};
use fparkan_nres::{decode as decode_nres, ReadProfile};
use fparkan_path::{normalize_relative, NormalizedPath, PathPolicy, ResourceName};
use fparkan_prototype::{EffectivePrototype, PrototypeGeometry, PrototypeGraph};
use fparkan_resource::{ResourceError, ResourceKey, ResourceRepository};
use fparkan_texm::decode_texm;
use std::collections::BTreeSet;
use std::fmt;
use std::hash::{Hash, Hasher};
use std::marker::PhantomData;
use std::sync::Arc;
const TEXTURES_ARCHIVE: &str = "textures.lib";
const LIGHTMAP_ARCHIVE: &str = "lightmap.lib";
/// Stable typed identifier for a prepared asset.
#[derive(Clone, Copy, Debug, Eq, PartialEq, Hash)]
pub struct AssetId<T> {
raw: u64,
marker: PhantomData<T>,
}
impl<T> AssetId<T> {
/// Creates an asset id from a stable raw value.
#[must_use]
pub const fn new(raw: u64) -> Self {
Self {
raw,
marker: PhantomData,
}
}
/// Returns the stable raw id.
#[must_use]
pub const fn raw(self) -> u64 {
self.raw
}
}
/// CPU-side data needed before a visual can be handed to a renderer.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct PreparedVisual {
/// Stable id derived from the prototype geometry key.
pub id: AssetId<PreparedVisual>,
/// Optional mesh resource backing the visual.
pub mesh: Option<ResourceKey>,
/// Number of validated model nodes.
pub model_nodes: usize,
/// Number of validated material slots on the model.
pub model_slots: usize,
/// Number of validated render batches.
pub model_batches: usize,
/// Number of WEAR material slots resolved through MAT0.
pub material_count: usize,
/// Number of texture phase requests decoded as TEXM.
pub texture_count: usize,
/// Number of lightmap requests decoded as TEXM.
pub lightmap_count: usize,
}
/// A transactional mission asset preparation plan.
#[derive(Clone, Debug, Default, Eq, PartialEq)]
pub struct MissionAssetPlan {
/// Number of visual prototypes in the plan.
pub visual_count: usize,
/// Number of mesh-backed visuals.
pub model_count: usize,
/// Number of material slot requests.
pub material_count: usize,
/// Number of texture phase requests.
pub texture_count: usize,
/// Number of lightmap requests.
pub lightmap_count: usize,
}
/// Coarse CPU-side asset budgets.
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
pub struct AssetBudgets {
/// Bytes parsed from source resource payloads.
pub parsed_bytes: u64,
}
/// Errors raised while preparing CPU-side assets.
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum AssetError {
/// A required cross-resource dependency was not found.
MissingDependency(String),
/// A prototype did not describe a usable visual.
InvalidPrototype(String),
/// A repository operation failed.
Resource(String),
/// MSH parsing or validation failed.
Msh(String),
/// WEAR/MAT0 parsing or resolution failed.
Material(String),
/// TEXM parsing failed.
Texture(String),
}
impl fmt::Display for AssetError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::MissingDependency(value) => write!(f, "missing dependency: {value}"),
Self::InvalidPrototype(value) => write!(f, "invalid prototype: {value}"),
Self::Resource(value) => write!(f, "resource error: {value}"),
Self::Msh(value) => write!(f, "msh error: {value}"),
Self::Material(value) => write!(f, "material error: {value}"),
Self::Texture(value) => write!(f, "texture error: {value}"),
}
}
}
impl std::error::Error for AssetError {}
/// Port implemented by typed asset loaders.
pub trait AssetLoader<T> {
/// Loads an asset for the given resource key.
///
/// # Errors
///
/// Returns [`AssetError`] when the resource cannot be resolved or decoded.
fn load(&self, key: &ResourceKey) -> Result<Arc<T>, AssetError>;
}
/// Minimal asset manager façade over an immutable resource repository.
#[derive(Debug)]
pub struct AssetManager<R> {
repository: R,
}
impl<R> AssetManager<R> {
/// Creates a manager backed by the given repository.
#[must_use]
pub const fn new(repository: R) -> Self {
Self { repository }
}
/// Returns the backing repository.
#[must_use]
pub const fn repository(&self) -> &R {
&self.repository
}
}
impl<R: ResourceRepository> AssetManager<R> {
/// Prepares one prototype visual using the manager repository.
///
/// # Errors
///
/// Returns [`AssetError`] if any model, material, texture, or lightmap
/// dependency is missing or malformed.
pub fn prepare_visual(&self, proto: &EffectivePrototype) -> Result<PreparedVisual, AssetError> {
prepare_visual_with_repository(&self.repository, proto)
}
/// Builds a mission plan by preparing each resolved prototype.
///
/// # Errors
///
/// Returns [`AssetError`] if any visual dependency is missing or malformed.
pub fn build_mission_asset_plan<'a>(
&self,
prototypes: impl IntoIterator<Item = &'a EffectivePrototype>,
) -> Result<MissionAssetPlan, AssetError> {
build_mission_asset_plan_with_repository(&self.repository, prototypes)
}
}
/// Produces a count-only plan from a prototype graph.
#[must_use]
pub fn build_mission_asset_plan(graph: &PrototypeGraph) -> MissionAssetPlan {
MissionAssetPlan {
visual_count: graph.prototype_requests.len(),
..MissionAssetPlan::default()
}
}
/// Builds a fully validated CPU-side mission asset plan.
///
/// # Errors
///
/// Returns [`AssetError`] if any reachable visual dependency is missing or
/// malformed.
pub fn build_mission_asset_plan_with_repository<'a, R: ResourceRepository>(
repository: &R,
prototypes: impl IntoIterator<Item = &'a EffectivePrototype>,
) -> Result<MissionAssetPlan, AssetError> {
let mut plan = MissionAssetPlan::default();
let mut prepared_visuals = BTreeSet::new();
for proto in prototypes {
let visual_id = stable_visual_id(proto);
if !prepared_visuals.insert(visual_id) {
continue;
}
let visual = prepare_visual_with_repository(repository, proto)?;
plan.visual_count += 1;
if visual.mesh.is_some() {
plan.model_count += 1;
}
plan.material_count += visual.material_count;
plan.texture_count += visual.texture_count;
plan.lightmap_count += visual.lightmap_count;
}
Ok(plan)
}
/// Validates a prototype visual without resolving cross-resource dependencies.
///
/// This is useful for tests and API callers that only need a stable visual id.
///
/// # Errors
///
/// Returns [`AssetError`] when the prototype geometry is malformed.
pub fn prepare_visual(proto: &EffectivePrototype) -> Result<PreparedVisual, AssetError> {
let id = stable_visual_id(proto);
let mesh = match &proto.geometry {
PrototypeGeometry::Mesh(key) => Some(key.clone()),
PrototypeGeometry::NonGeometric => None,
};
Ok(PreparedVisual {
id: AssetId::new(id),
mesh,
model_nodes: 0,
model_slots: 0,
model_batches: 0,
material_count: 0,
texture_count: 0,
lightmap_count: 0,
})
}
/// Prepares one visual and validates all CPU-side resource dependencies.
///
/// # Errors
///
/// Returns [`AssetError`] if the model, WEAR table, MAT0 materials, texture
/// phases, or lightmaps cannot be resolved and decoded.
pub fn prepare_visual_with_repository<R: ResourceRepository>(
repository: &R,
proto: &EffectivePrototype,
) -> Result<PreparedVisual, AssetError> {
let PrototypeGeometry::Mesh(mesh_key) = &proto.geometry else {
return prepare_visual(proto);
};
let nres = decode_nres(
read_key(repository, mesh_key, Some("mesh"))?,
ReadProfile::Compatible,
)
.map_err(|err| AssetError::Msh(err.to_string()))?;
let msh_document = decode_msh(&nres).map_err(|err| AssetError::Msh(err.to_string()))?;
let model = validate_msh(&msh_document).map_err(|err| AssetError::Msh(err.to_string()))?;
let wear_name = sibling_name(mesh_key, "wea")?;
let wear_key = ResourceKey {
archive: mesh_key.archive.clone(),
name: wear_name,
type_id: Some(WEAR_KIND),
};
let wear = decode_wear(&read_key(repository, &wear_key, Some("wear"))?)
.map_err(|err| AssetError::Material(err.to_string()))?;
let mut material_count = 0;
let mut texture_count = 0;
let mut lightmap_count = 0;
for material_index in 0..wear.entries.len() {
let material_index = u16::try_from(material_index).map_err(|_| {
AssetError::Material("material index does not fit archive format".to_string())
})?;
let material = resolve_material(repository, &wear, material_index)
.map_err(|err| AssetError::Material(err.to_string()))?;
material_count += 1;
for texture in material.document.texture_requests() {
resolve_texture(repository, &texture)?;
texture_count += 1;
}
}
for lightmap in &wear.lightmaps {
resolve_lightmap(repository, &lightmap.lightmap)?;
lightmap_count += 1;
}
Ok(PreparedVisual {
id: AssetId::new(stable_visual_id(proto)),
mesh: Some(mesh_key.clone()),
model_nodes: model.node_count,
model_slots: model.slots.len(),
model_batches: model.batches.len(),
material_count,
texture_count,
lightmap_count,
})
}
fn read_key<R: ResourceRepository>(
repository: &R,
key: &ResourceKey,
label: Option<&str>,
) -> Result<Arc<[u8]>, AssetError> {
let handle = repository
.open_archive(&key.archive)
.map_err(|err| AssetError::Resource(format!("{label:?} {key:?}: {err}")))
.and_then(|archive| {
repository
.find(archive, &key.name)
.map_err(|err| AssetError::Resource(format!("{label:?} {key:?}: {err}")))
})?
.ok_or_else(|| AssetError::MissingDependency(format!("{label:?} {key:?}")))?;
let bytes = repository
.read(handle)
.map_err(|err| AssetError::Resource(format!("{label:?} {key:?}: {err}")))?;
Ok(Arc::from(bytes.into_owned()))
}
fn resolve_texture<R: ResourceRepository>(
repository: &R,
name: &ResourceName,
) -> Result<(), AssetError> {
resolve_texm(repository, name, TEXTURES_ARCHIVE, "texture")
}
fn resolve_lightmap<R: ResourceRepository>(
repository: &R,
name: &ResourceName,
) -> Result<(), AssetError> {
resolve_texm(repository, name, LIGHTMAP_ARCHIVE, "lightmap")
}
fn resolve_texm<R: ResourceRepository>(
repository: &R,
name: &ResourceName,
archive: &str,
label: &'static str,
) -> Result<(), AssetError> {
let key = ResourceKey {
archive: parse_path(archive)?,
name: name.clone(),
type_id: None,
};
let Some(bytes) = read_optional_key(repository, &key, Some(label))? else {
return Err(AssetError::MissingDependency(format!("{label} {name:?}")));
};
decode_texm(bytes)
.map(|_| ())
.map_err(|err| AssetError::Texture(err.to_string()))
}
fn read_optional_key<R: ResourceRepository>(
repository: &R,
key: &ResourceKey,
label: Option<&str>,
) -> Result<Option<Arc<[u8]>>, AssetError> {
let archive = match repository.open_archive(&key.archive) {
Ok(archive) => archive,
Err(ResourceError::MissingArchive | ResourceError::MissingEntry) => return Ok(None),
Err(err) => return Err(AssetError::Resource(format!("{label:?} {key:?}: {err}"))),
};
let Some(handle) = repository
.find(archive, &key.name)
.map_err(|err| AssetError::Resource(format!("{label:?} {key:?}: {err}")))?
else {
return Ok(None);
};
let bytes = repository
.read(handle)
.map_err(|err| AssetError::Resource(format!("{label:?} {key:?}: {err}")))?;
Ok(Some(Arc::from(bytes.into_owned())))
}
fn sibling_name(key: &ResourceKey, extension: &str) -> Result<ResourceName, AssetError> {
let dot = key
.name
.0
.iter()
.rposition(|byte| *byte == b'.')
.ok_or_else(|| {
AssetError::InvalidPrototype(format!("resource name has no extension: {:?}", key.name))
})?;
let mut name = key.name.0[..dot].to_vec();
name.push(b'.');
name.extend_from_slice(extension.as_bytes());
Ok(ResourceName(name))
}
fn stable_visual_id(proto: &EffectivePrototype) -> u64 {
let mut hasher = StableHasher::default();
match &proto.geometry {
PrototypeGeometry::Mesh(key) => {
1_u8.hash(&mut hasher);
key.archive.as_str().hash(&mut hasher);
key.name.0.hash(&mut hasher);
key.type_id.hash(&mut hasher);
}
PrototypeGeometry::NonGeometric => {
0_u8.hash(&mut hasher);
}
}
hasher.finish()
}
fn parse_path(value: &str) -> Result<NormalizedPath, AssetError> {
normalize_relative(value.as_bytes(), PathPolicy::HostCompatible)
.map_err(|err| AssetError::InvalidPrototype(format!("{err}")))
}
#[derive(Default)]
struct StableHasher(u64);
impl Hasher for StableHasher {
fn finish(&self) -> u64 {
self.0
}
fn write(&mut self, bytes: &[u8]) {
let mut value = if self.0 == 0 {
0xcbf2_9ce4_8422_2325
} else {
self.0
};
for byte in bytes {
value ^= u64::from(*byte);
value = value.wrapping_mul(0x0000_0100_0000_01b3);
}
self.0 = value;
}
}
#[cfg(test)]
mod tests {
use super::*;
use fparkan_prototype::build_prototype_graph;
use fparkan_resource::{resource_name, CachedResourceRepository};
use fparkan_vfs::{DirectoryVfs, MemoryVfs, Vfs};
use std::path::PathBuf;
#[test]
fn count_only_plan_uses_graph_requests() {
let graph = PrototypeGraph::default();
let plan = build_mission_asset_plan(&graph);
assert_eq!(plan.visual_count, 0);
assert_eq!(plan.model_count, 0);
}
#[test]
fn texture_resolver_does_not_fallback_to_lightmap_archive() {
let texm = texm_payload();
let repo = repository_with_archives(&[(
LIGHTMAP_ARCHIVE,
&[(b"TEX_ONLY".as_slice(), texm.as_slice())],
)]);
let err = resolve_texture(&repo, &resource_name(b"TEX_ONLY")).expect_err("missing texture");
assert!(matches!(err, AssetError::MissingDependency(_)));
}
#[test]
fn lightmap_resolver_does_not_fallback_to_texture_archive() {
let texm = texm_payload();
let repo = repository_with_archives(&[(
TEXTURES_ARCHIVE,
&[(b"LM_ONLY".as_slice(), texm.as_slice())],
)]);
let err =
resolve_lightmap(&repo, &resource_name(b"LM_ONLY")).expect_err("missing lightmap");
assert!(matches!(err, AssetError::MissingDependency(_)));
}
#[test]
fn texture_resolver_does_not_continue_after_malformed_texture() {
let malformed = b"not texm".as_slice();
let texm = texm_payload();
let repo = repository_with_archives(&[
(TEXTURES_ARCHIVE, &[(b"BAD".as_slice(), malformed)]),
(LIGHTMAP_ARCHIVE, &[(b"BAD".as_slice(), texm.as_slice())]),
]);
let err = resolve_texture(&repo, &resource_name(b"BAD")).expect_err("malformed texture");
assert!(matches!(err, AssetError::Texture(_)));
}
#[test]
#[ignore = "requires licensed corpus"]
fn prepares_real_unit_asset_plan() {
let root = fixture_root("IS");
let vfs: Arc<dyn Vfs> = Arc::new(DirectoryVfs::new(&root));
let repository = CachedResourceRepository::new(Arc::clone(&vfs));
let roots = [resource_name(b"UNITS/AUTO/swlklas.dat")];
let (graph, prototypes) =
build_prototype_graph(&repository, vfs.as_ref(), &roots).expect("prototype graph");
let count_only = build_mission_asset_plan(&graph);
let plan = build_mission_asset_plan_with_repository(&repository, &prototypes)
.expect("asset preparation");
assert_eq!(count_only.visual_count, 12);
assert_eq!(prototypes.len(), 12);
assert_eq!(plan.visual_count, 11);
assert_eq!(plan.model_count, 11);
assert_eq!(plan.material_count, 62);
assert_eq!(plan.texture_count, 77);
assert_eq!(plan.lightmap_count, 0);
}
#[test]
#[ignore = "requires licensed corpus"]
fn repository_plan_deduplicates_duplicate_visuals_but_graph_preserves_requests() {
let root = fixture_root("IS");
let vfs: Arc<dyn Vfs> = Arc::new(DirectoryVfs::new(&root));
let repository = CachedResourceRepository::new(Arc::clone(&vfs));
let roots = [
resource_name(b"UNITS/AUTO/swlklas.dat"),
resource_name(b"UNITS/AUTO/swlklas.dat"),
];
let (graph, prototypes) =
build_prototype_graph(&repository, vfs.as_ref(), &roots).expect("prototype graph");
let count_only = build_mission_asset_plan(&graph);
let plan = build_mission_asset_plan_with_repository(&repository, &prototypes)
.expect("asset preparation");
assert_eq!(graph.roots.len(), 2);
assert_eq!(count_only.visual_count, 24);
assert_eq!(prototypes.len(), 24);
assert_eq!(plan.visual_count, 11);
assert_eq!(plan.model_count, 11);
assert_eq!(plan.material_count, 62);
assert_eq!(plan.texture_count, 77);
}
fn fixture_root(part: &str) -> PathBuf {
let variable = match part {
"IS" => "FPARKAN_CORPUS_PART1_ROOT",
"IS2" => "FPARKAN_CORPUS_PART2_ROOT",
_ => panic!("unknown licensed corpus part: {part}"),
};
let root = std::env::var_os(variable)
.map(PathBuf::from)
.unwrap_or_else(|| panic!("{variable} is required for licensed corpus tests"));
assert!(
root.is_dir(),
"licensed corpus root is missing: {}",
root.display()
);
root
}
fn repository_with_archives(
archives: &[(&str, &[(&[u8], &[u8])])],
) -> CachedResourceRepository {
let mut vfs = MemoryVfs::default();
for (archive, entries) in archives {
let path = parse_path(archive).expect("archive path");
vfs.insert(path, Arc::from(build_nres(entries).into_boxed_slice()));
}
CachedResourceRepository::new(Arc::new(vfs))
}
fn texm_payload() -> Vec<u8> {
let mut out = Vec::new();
out.extend_from_slice(&0x6d78_6554_u32.to_le_bytes());
out.extend_from_slice(&1_u32.to_le_bytes());
out.extend_from_slice(&1_u32.to_le_bytes());
out.extend_from_slice(&1_u32.to_le_bytes());
out.extend_from_slice(&0_u32.to_le_bytes());
out.extend_from_slice(&0_u32.to_le_bytes());
out.extend_from_slice(&0_u32.to_le_bytes());
out.extend_from_slice(&565_u32.to_le_bytes());
out.extend_from_slice(&0xffff_u16.to_le_bytes());
out
}
fn build_nres(entries: &[(&[u8], &[u8])]) -> Vec<u8> {
let mut out = vec![0; 16];
let mut offsets = Vec::with_capacity(entries.len());
for (_, payload) in entries {
offsets.push(u32::try_from(out.len()).expect("offset"));
out.extend_from_slice(payload);
let padding = (8 - (out.len() % 8)) % 8;
out.resize(out.len() + padding, 0);
}
let mut order: Vec<usize> = (0..entries.len()).collect();
order.sort_by(|left, right| entries[*left].0.cmp(entries[*right].0));
for (idx, (name, payload)) in entries.iter().enumerate() {
push_u32(&mut out, 0);
push_u32(&mut out, 0);
push_u32(&mut out, 0);
push_u32(&mut out, u32::try_from(payload.len()).expect("payload"));
push_u32(&mut out, 0);
let mut name_raw = [0; 36];
let len = name_raw.len().saturating_sub(1).min(name.len());
name_raw[..len].copy_from_slice(&name[..len]);
out.extend_from_slice(&name_raw);
push_u32(&mut out, offsets[idx]);
push_u32(&mut out, u32::try_from(order[idx]).expect("sort index"));
}
out[0..4].copy_from_slice(b"NRes");
out[4..8].copy_from_slice(&0x100_u32.to_le_bytes());
out[8..12].copy_from_slice(&u32::try_from(entries.len()).expect("count").to_le_bytes());
let total_size = u32::try_from(out.len()).expect("total size");
out[12..16].copy_from_slice(&total_size.to_le_bytes());
out
}
fn push_u32(out: &mut Vec<u8>, value: u32) {
out.extend_from_slice(&value.to_le_bytes());
}
}
+11
View File
@@ -0,0 +1,11 @@
[package]
name = "fparkan-binary"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
[lints]
workspace = true
+500
View File
@@ -0,0 +1,500 @@
#![forbid(unsafe_code)]
//! Bounded little-endian binary cursor and checked layout helpers.
use std::fmt;
/// SHA-256 digest bytes.
pub type Sha256Digest = [u8; 32];
/// Parser limits shared by binary formats.
#[derive(Clone, Copy, Debug)]
pub struct Limits {
/// Maximum file bytes.
pub max_file_bytes: u64,
/// Maximum entries.
pub max_entries: u32,
/// Maximum string bytes.
pub max_string_bytes: u32,
/// Maximum array items.
pub max_array_items: u32,
/// Maximum recursion depth.
pub max_recursion_depth: u16,
/// Maximum decoded bytes.
pub max_decoded_bytes: u64,
}
impl Default for Limits {
fn default() -> Self {
Self {
max_file_bytes: 256 * 1024 * 1024,
max_entries: 1_000_000,
max_string_bytes: 64 * 1024,
max_array_items: 1_000_000,
max_recursion_depth: 64,
max_decoded_bytes: 512 * 1024 * 1024,
}
}
}
/// Decode error.
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum DecodeError {
/// Input ended before requested bytes.
UnexpectedEof {
/// Offset where read was attempted.
offset: u64,
/// Required byte count.
needed: u64,
/// Remaining byte count.
remaining: u64,
},
/// Arithmetic overflow.
IntegerOverflow,
/// Count exceeds limit.
LimitExceeded {
/// Declared count.
count: u64,
/// Configured limit.
limit: u64,
},
/// Cursor did not end at EOF.
TrailingBytes {
/// Offset where EOF was expected.
offset: u64,
/// Remaining byte count.
remaining: u64,
},
/// Invalid data.
Invalid(&'static str),
}
impl fmt::Display for DecodeError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::UnexpectedEof {
offset,
needed,
remaining,
} => write!(
f,
"unexpected EOF at {offset}: need {needed}, have {remaining}"
),
Self::IntegerOverflow => write!(f, "integer overflow"),
Self::LimitExceeded { count, limit } => {
write!(f, "count {count} exceeds limit {limit}")
}
Self::TrailingBytes { offset, remaining } => {
write!(f, "trailing bytes at {offset}: {remaining}")
}
Self::Invalid(reason) => write!(f, "invalid data: {reason}"),
}
}
}
impl std::error::Error for DecodeError {}
/// Cursor checkpoint.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct Checkpoint(pub u64);
/// Bounded cursor.
#[derive(Clone, Debug)]
pub struct Cursor<'a> {
bytes: &'a [u8],
offset: usize,
}
impl<'a> Cursor<'a> {
/// Creates a cursor.
#[must_use]
pub fn new(bytes: &'a [u8]) -> Self {
Self { bytes, offset: 0 }
}
/// Current offset.
#[must_use]
pub fn offset(&self) -> u64 {
self.offset as u64
}
/// Remaining bytes.
#[must_use]
pub fn remaining(&self) -> usize {
self.bytes.len().saturating_sub(self.offset)
}
/// Creates a checkpoint.
#[must_use]
pub fn checkpoint(&self) -> Checkpoint {
Checkpoint(self.offset())
}
/// Reads exact bytes.
///
/// # Errors
///
/// Returns [`DecodeError::IntegerOverflow`] if the requested end offset
/// overflows, or [`DecodeError::UnexpectedEof`] if there are not enough
/// bytes remaining.
pub fn read_exact(&mut self, len: usize) -> Result<&'a [u8], DecodeError> {
let end = self
.offset
.checked_add(len)
.ok_or(DecodeError::IntegerOverflow)?;
if end > self.bytes.len() {
return Err(DecodeError::UnexpectedEof {
offset: self.offset(),
needed: len as u64,
remaining: self.remaining() as u64,
});
}
let out = &self.bytes[self.offset..end];
self.offset = end;
Ok(out)
}
/// Reads a little-endian u16.
///
/// # Errors
///
/// Returns [`DecodeError`] if two bytes cannot be read.
pub fn read_u16_le(&mut self) -> Result<u16, DecodeError> {
let b = self.read_exact(2)?;
Ok(u16::from_le_bytes([b[0], b[1]]))
}
/// Reads a little-endian u32.
///
/// # Errors
///
/// Returns [`DecodeError`] if four bytes cannot be read.
pub fn read_u32_le(&mut self) -> Result<u32, DecodeError> {
let b = self.read_exact(4)?;
Ok(u32::from_le_bytes([b[0], b[1], b[2], b[3]]))
}
/// Reads a little-endian i32.
///
/// # Errors
///
/// Returns [`DecodeError`] if four bytes cannot be read.
pub fn read_i32_le(&mut self) -> Result<i32, DecodeError> {
let b = self.read_exact(4)?;
Ok(i32::from_le_bytes([b[0], b[1], b[2], b[3]]))
}
/// Reads a little-endian f32.
///
/// # Errors
///
/// Returns [`DecodeError`] if four bytes cannot be read.
pub fn read_f32_le(&mut self) -> Result<f32, DecodeError> {
Ok(f32::from_bits(self.read_u32_le()?))
}
/// Requires exact EOF.
///
/// # Errors
///
/// Returns [`DecodeError::TrailingBytes`] when unread bytes remain.
pub fn require_eof(&self) -> Result<(), DecodeError> {
if self.remaining() == 0 {
Ok(())
} else {
Err(DecodeError::TrailingBytes {
offset: self.offset(),
remaining: self.remaining() as u64,
})
}
}
}
/// Validates `count * stride <= remaining` and returns bytes as usize.
///
/// # Errors
///
/// Returns [`DecodeError::IntegerOverflow`] on arithmetic or conversion
/// overflow, or [`DecodeError::UnexpectedEof`] when the declared byte count is
/// larger than the remaining bounded input.
pub fn checked_count_bytes(count: u64, stride: u64, remaining: u64) -> Result<usize, DecodeError> {
let bytes = count
.checked_mul(stride)
.ok_or(DecodeError::IntegerOverflow)?;
if bytes > remaining {
return Err(DecodeError::UnexpectedEof {
offset: 0,
needed: bytes,
remaining,
});
}
usize::try_from(bytes).map_err(|_| DecodeError::IntegerOverflow)
}
/// Validates a declared allocation size before constructing the allocation.
///
/// # Errors
///
/// Returns [`DecodeError::LimitExceeded`] when `declared` is larger than
/// `limit`, or [`DecodeError::IntegerOverflow`] when the accepted size cannot
/// be represented by the host `usize`.
pub fn checked_allocation_len(declared: u64, limit: u64) -> Result<usize, DecodeError> {
if declared > limit {
return Err(DecodeError::LimitExceeded {
count: declared,
limit,
});
}
usize::try_from(declared).map_err(|_| DecodeError::IntegerOverflow)
}
/// Reads length-prefixed bytes.
///
/// # Errors
///
/// Returns [`DecodeError`] if the length cannot be read, exceeds `max`, or the
/// declared payload is truncated.
pub fn read_lp_bytes(cursor: &mut Cursor<'_>, max: u32) -> Result<Vec<u8>, DecodeError> {
let len = cursor.read_u32_le()?;
if len > max {
return Err(DecodeError::LimitExceeded {
count: u64::from(len),
limit: u64::from(max),
});
}
let len = checked_allocation_len(u64::from(len), u64::from(max))?;
Ok(cursor.read_exact(len)?.to_vec())
}
/// Computes a SHA-256 content digest without external dependencies.
#[must_use]
pub fn sha256(bytes: &[u8]) -> Sha256Digest {
const K: [u32; 64] = [
0x428a_2f98,
0x7137_4491,
0xb5c0_fbcf,
0xe9b5_dba5,
0x3956_c25b,
0x59f1_11f1,
0x923f_82a4,
0xab1c_5ed5,
0xd807_aa98,
0x1283_5b01,
0x2431_85be,
0x550c_7dc3,
0x72be_5d74,
0x80de_b1fe,
0x9bdc_06a7,
0xc19b_f174,
0xe49b_69c1,
0xefbe_4786,
0x0fc1_9dc6,
0x240c_a1cc,
0x2de9_2c6f,
0x4a74_84aa,
0x5cb0_a9dc,
0x76f9_88da,
0x983e_5152,
0xa831_c66d,
0xb003_27c8,
0xbf59_7fc7,
0xc6e0_0bf3,
0xd5a7_9147,
0x06ca_6351,
0x1429_2967,
0x27b7_0a85,
0x2e1b_2138,
0x4d2c_6dfc,
0x5338_0d13,
0x650a_7354,
0x766a_0abb,
0x81c2_c92e,
0x9272_2c85,
0xa2bf_e8a1,
0xa81a_664b,
0xc24b_8b70,
0xc76c_51a3,
0xd192_e819,
0xd699_0624,
0xf40e_3585,
0x106a_a070,
0x19a4_c116,
0x1e37_6c08,
0x2748_774c,
0x34b0_bcb5,
0x391c_0cb3,
0x4ed8_aa4a,
0x5b9c_ca4f,
0x682e_6ff3,
0x748f_82ee,
0x78a5_636f,
0x84c8_7814,
0x8cc7_0208,
0x90be_fffa,
0xa450_6ceb,
0xbef9_a3f7,
0xc671_78f2,
];
let mut h = [
0x6a09_e667,
0xbb67_ae85,
0x3c6e_f372,
0xa54f_f53a,
0x510e_527f,
0x9b05_688c,
0x1f83_d9ab,
0x5be0_cd19,
];
let bit_len = (bytes.len() as u64).wrapping_mul(8);
let mut chunks = bytes.chunks_exact(64);
for chunk in &mut chunks {
compress_sha256_chunk(&mut h, chunk, &K);
}
let tail = chunks.remainder();
let mut block = [0u8; 128];
block[..tail.len()].copy_from_slice(tail);
block[tail.len()] = 0x80;
let padded_len = if tail.len() < 56 { 64 } else { 128 };
block[padded_len - 8..padded_len].copy_from_slice(&bit_len.to_be_bytes());
for chunk in block[..padded_len].chunks_exact(64) {
compress_sha256_chunk(&mut h, chunk, &K);
}
let mut out = [0u8; 32];
for (idx, word) in h.iter().enumerate() {
out[idx * 4..idx * 4 + 4].copy_from_slice(&word.to_be_bytes());
}
out
}
/// Renders a SHA-256 digest as lowercase hexadecimal.
#[must_use]
pub fn sha256_hex(digest: &Sha256Digest) -> String {
const HEX: &[u8; 16] = b"0123456789abcdef";
let mut out = String::with_capacity(64);
for byte in digest {
out.push(char::from(HEX[usize::from(byte >> 4)]));
out.push(char::from(HEX[usize::from(byte & 0x0f)]));
}
out
}
#[allow(clippy::many_single_char_names)]
fn compress_sha256_chunk(h: &mut [u32; 8], chunk: &[u8], k: &[u32; 64]) {
let mut w = [0u32; 64];
for (idx, word) in w.iter_mut().take(16).enumerate() {
let base = idx * 4;
*word = u32::from_be_bytes([
chunk[base],
chunk[base + 1],
chunk[base + 2],
chunk[base + 3],
]);
}
for idx in 16..64 {
let s0 = w[idx - 15].rotate_right(7) ^ w[idx - 15].rotate_right(18) ^ (w[idx - 15] >> 3);
let s1 = w[idx - 2].rotate_right(17) ^ w[idx - 2].rotate_right(19) ^ (w[idx - 2] >> 10);
w[idx] = w[idx - 16]
.wrapping_add(s0)
.wrapping_add(w[idx - 7])
.wrapping_add(s1);
}
let mut a = h[0];
let mut b = h[1];
let mut c = h[2];
let mut d = h[3];
let mut e = h[4];
let mut f = h[5];
let mut g = h[6];
let mut hh = h[7];
for idx in 0..64 {
let s1 = e.rotate_right(6) ^ e.rotate_right(11) ^ e.rotate_right(25);
let ch = (e & f) ^ ((!e) & g);
let temp1 = hh
.wrapping_add(s1)
.wrapping_add(ch)
.wrapping_add(k[idx])
.wrapping_add(w[idx]);
let s0 = a.rotate_right(2) ^ a.rotate_right(13) ^ a.rotate_right(22);
let maj = (a & b) ^ (a & c) ^ (b & c);
let temp2 = s0.wrapping_add(maj);
hh = g;
g = f;
f = e;
e = d.wrapping_add(temp1);
d = c;
c = b;
b = a;
a = temp1.wrapping_add(temp2);
}
h[0] = h[0].wrapping_add(a);
h[1] = h[1].wrapping_add(b);
h[2] = h[2].wrapping_add(c);
h[3] = h[3].wrapping_add(d);
h[4] = h[4].wrapping_add(e);
h[5] = h[5].wrapping_add(f);
h[6] = h[6].wrapping_add(g);
h[7] = h[7].wrapping_add(hh);
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn rejects_count_stride_overflow() {
assert_eq!(
checked_count_bytes(u64::MAX, 2, u64::MAX),
Err(DecodeError::IntegerOverflow)
);
}
#[test]
fn exact_eof_reports_trailing() {
let mut cursor = Cursor::new(&[1, 2]);
assert_eq!(cursor.read_exact(1).expect("byte"), &[1]);
assert!(matches!(
cursor.require_eof(),
Err(DecodeError::TrailingBytes { .. })
));
}
#[test]
fn rejects_oversized_declared_allocation_before_read() {
assert_eq!(
checked_allocation_len(1025, 1024),
Err(DecodeError::LimitExceeded {
count: 1025,
limit: 1024
})
);
let bytes = 2048u32.to_le_bytes();
let mut cursor = Cursor::new(&bytes);
assert_eq!(
read_lp_bytes(&mut cursor, 1024),
Err(DecodeError::LimitExceeded {
count: 2048,
limit: 1024
})
);
assert_eq!(cursor.offset(), 4);
}
#[test]
fn sha256_matches_known_vectors() {
assert_eq!(
sha256_hex(&sha256(b"")),
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
);
assert_eq!(
sha256_hex(&sha256(b"abc")),
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"
);
}
}
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "fparkan-corpus"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-binary = { path = "../fparkan-binary" }
fparkan-nres = { path = "../fparkan-nres" }
fparkan-path = { path = "../fparkan-path" }
[lints]
workspace = true
+924
View File
@@ -0,0 +1,924 @@
#![forbid(unsafe_code)]
//! Licensed corpus discovery and aggregate reports.
use fparkan_binary::{sha256, sha256_hex, Sha256Digest};
use fparkan_path::{ascii_lookup_key, normalize_relative, PathPolicy};
use std::collections::{BTreeMap, BTreeSet};
use std::fmt;
use std::fs;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::Arc;
/// Corpus kind.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum CorpusKind {
/// Demo corpus.
Demo,
/// Part 1 full game.
Part1,
/// Part 2 full game.
Part2,
/// Unknown local directory.
Unknown,
}
/// Corpus root.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct CorpusRoot(pub PathBuf);
/// Discovery options.
#[derive(Clone, Copy, Debug, Default)]
pub struct DiscoverOptions {
/// Whether symlinks may be traversed.
pub follow_symlinks: bool,
}
/// File manifest entry.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct ManifestEntry {
/// Normalized relative path.
pub path: String,
/// File size in bytes.
pub size: u64,
/// SHA-256 content fingerprint.
pub hash: Sha256Digest,
}
/// Corpus manifest.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct CorpusManifest {
/// Kind.
pub kind: CorpusKind,
/// Sorted files.
pub files: Vec<ManifestEntry>,
/// Casefold collisions.
pub casefold_collisions: Vec<Vec<String>>,
}
/// Aggregate report.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct CorpusReport {
/// Schema version.
pub schema: u32,
/// Kind.
pub kind: CorpusKind,
/// Total files.
pub files: usize,
/// Total bytes.
pub bytes: u64,
/// Metrics.
pub metrics: BTreeMap<String, u64>,
/// Casefold collision count.
pub casefold_collisions: usize,
/// Manifest fingerprint.
pub fingerprint: Sha256Digest,
/// Per-file status records.
pub records: Vec<CorpusFileRecord>,
/// Number of files with report errors.
pub failures: usize,
}
/// Per-file report status.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum CorpusFileStatus {
/// File was inspected successfully.
Ok,
/// File was inspected but produced a non-fatal warning.
Warning,
/// File could not be inspected.
Error,
}
/// Per-file report record.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct CorpusFileRecord {
/// Normalized relative path.
pub path: String,
/// Inspection status.
pub status: CorpusFileStatus,
/// Detected file variant.
pub variant: String,
/// Optional status message.
pub message: Option<String>,
}
/// Corpus error.
#[derive(Debug)]
pub enum CorpusError {
/// I/O failure.
Io {
/// Path where I/O failed.
path: PathBuf,
/// Source error.
source: std::io::Error,
},
/// Invalid root.
InvalidRoot(PathBuf),
/// Invalid path.
InvalidPath(String),
/// Aggregate report failure.
Report {
/// Path where reporting failed.
path: String,
/// Failure message.
message: String,
},
}
impl fmt::Display for CorpusError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Io { path, source } => write!(f, "{}: {source}", path.display()),
Self::InvalidRoot(path) => write!(f, "invalid corpus root: {}", path.display()),
Self::InvalidPath(path) => write!(f, "invalid corpus path: {path}"),
Self::Report { path, message } => write!(f, "{path}: {message}"),
}
}
}
impl std::error::Error for CorpusError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
Self::Io { source, .. } => Some(source),
Self::InvalidRoot(_) | Self::InvalidPath(_) | Self::Report { .. } => None,
}
}
}
/// Discovers a corpus under a root directory.
///
/// # Errors
///
/// Returns [`CorpusError`] if the root is invalid, traversal encounters an I/O
/// error, or a discovered path cannot be represented by the legacy path policy.
pub fn discover(root: &Path, options: DiscoverOptions) -> Result<CorpusManifest, CorpusError> {
if !root.is_dir() {
return Err(CorpusError::InvalidRoot(root.to_path_buf()));
}
let mut files = Vec::new();
walk(root, root, options, &mut files)?;
files.sort_by(|a, b| a.path.cmp(&b.path));
let kind = classify(root, &files);
let casefold_collisions = detect_casefold_collisions(&files);
Ok(CorpusManifest {
kind,
files,
casefold_collisions,
})
}
fn walk(
root: &Path,
dir: &Path,
options: DiscoverOptions,
out: &mut Vec<ManifestEntry>,
) -> Result<(), CorpusError> {
let read_dir = fs::read_dir(dir).map_err(|source| CorpusError::Io {
path: dir.to_path_buf(),
source,
})?;
let mut entries = Vec::new();
for entry in read_dir {
let entry = entry.map_err(|source| CorpusError::Io {
path: dir.to_path_buf(),
source,
})?;
entries.push(entry.path());
}
entries.sort();
for path in entries {
if path
.file_name()
.and_then(|name| name.to_str())
.is_some_and(|name| name.starts_with('.'))
{
continue;
}
let metadata = fs::symlink_metadata(&path).map_err(|source| CorpusError::Io {
path: path.clone(),
source,
})?;
if metadata.file_type().is_symlink() && !options.follow_symlinks {
continue;
}
if metadata.is_dir() {
walk(root, &path, options, out)?;
continue;
}
if !metadata.is_file() {
continue;
}
let rel = path
.strip_prefix(root)
.map_err(|_| CorpusError::InvalidPath(path.display().to_string()))?;
let rel_text = rel
.to_str()
.ok_or_else(|| CorpusError::InvalidPath(path.display().to_string()))?;
let normalized = normalize_relative(rel_text.as_bytes(), PathPolicy::HostCompatible)
.map_err(|_| CorpusError::InvalidPath(rel_text.to_string()))?;
let bytes = fs::read(&path).map_err(|source| CorpusError::Io {
path: path.clone(),
source,
})?;
out.push(ManifestEntry {
path: normalized.as_str().to_string(),
size: metadata.len(),
hash: sha256(&bytes),
});
}
Ok(())
}
fn classify(root: &Path, files: &[ManifestEntry]) -> CorpusKind {
let name = root
.file_name()
.and_then(|v| v.to_str())
.unwrap_or_default()
.to_ascii_uppercase();
if name == "IS" {
CorpusKind::Part1
} else if name == "IS2" {
CorpusKind::Part2
} else if files
.iter()
.any(|f| f.path.eq_ignore_ascii_case("iron_3d.exe"))
{
CorpusKind::Part1
} else {
CorpusKind::Unknown
}
}
fn detect_casefold_collisions(files: &[ManifestEntry]) -> Vec<Vec<String>> {
let mut grouped: BTreeMap<Vec<u8>, BTreeSet<String>> = BTreeMap::new();
for file in files {
grouped
.entry(ascii_lookup_key(file.path.as_bytes()).0)
.or_default()
.insert(file.path.clone());
}
grouped
.into_values()
.filter(|paths| paths.len() > 1)
.map(|paths| paths.into_iter().collect())
.collect()
}
/// Builds aggregate report.
///
/// # Errors
///
/// Returns [`CorpusError`] when the aggregate report cannot be constructed.
/// Per-file inspection failures are represented in [`CorpusReport::records`]
/// and counted in [`CorpusReport::failures`].
pub fn report(root: &Path, manifest: &CorpusManifest) -> Result<CorpusReport, CorpusError> {
let mut metrics = empty_report_metrics();
let mut records = Vec::with_capacity(manifest.files.len());
let mut failures = 0usize;
for entry in &manifest.files {
let record = inspect_report_file(root, entry, &mut metrics);
if record.status == CorpusFileStatus::Error {
failures = failures.saturating_add(1);
}
records.push(record);
}
Ok(CorpusReport {
schema: 1,
kind: manifest.kind,
files: manifest.files.len(),
bytes: manifest.files.iter().map(|f| f.size).sum(),
metrics,
casefold_collisions: manifest.casefold_collisions.len(),
fingerprint: fingerprint(manifest),
records,
failures,
})
}
fn empty_report_metrics() -> BTreeMap<String, u64> {
let mut metrics = BTreeMap::new();
metrics.insert("nres_files".to_string(), 0);
metrics.insert("nres_entries".to_string(), 0);
metrics.insert("rsli_files".to_string(), 0);
metrics.insert("tma_files".to_string(), 0);
metrics.insert("land_msh_files".to_string(), 0);
metrics.insert("land_map_files".to_string(), 0);
metrics.insert("unit_dat_files".to_string(), 0);
metrics.insert("msh_entries".to_string(), 0);
metrics.insert("mat0_entries".to_string(), 0);
metrics.insert("texm_entries".to_string(), 0);
metrics.insert("fxid_entries".to_string(), 0);
metrics.insert("wear_entries".to_string(), 0);
metrics
}
fn inspect_report_file(
root: &Path,
entry: &ManifestEntry,
metrics: &mut BTreeMap<String, u64>,
) -> CorpusFileRecord {
let lower = entry.path.to_ascii_lowercase();
let mut variant = inspect_path_metrics(&lower, metrics);
let path = root.join(&entry.path);
let bytes = match fs::read(&path) {
Ok(bytes) => bytes,
Err(source) => {
return CorpusFileRecord {
path: entry.path.clone(),
status: CorpusFileStatus::Error,
variant,
message: Some(source.to_string()),
};
}
};
if bytes.starts_with(b"NRes") {
variant = "nres".to_string();
bump(metrics, "nres_files", 1);
if let Err(message) = inspect_nres_metrics(bytes, metrics) {
return CorpusFileRecord {
path: entry.path.clone(),
status: CorpusFileStatus::Error,
variant,
message: Some(message),
};
}
} else if bytes.starts_with(b"NL") {
variant = "rsli".to_string();
bump(metrics, "rsli_files", 1);
}
CorpusFileRecord {
path: entry.path.clone(),
status: CorpusFileStatus::Ok,
variant,
message: None,
}
}
fn inspect_path_metrics(lower: &str, metrics: &mut BTreeMap<String, u64>) -> String {
let mut variant = "file";
if lower.ends_with("data.tma") {
bump(metrics, "tma_files", 1);
variant = "tma";
}
if lower.ends_with("land.msh") {
bump(metrics, "land_msh_files", 1);
variant = "land_msh";
}
if lower.ends_with("land.map") {
bump(metrics, "land_map_files", 1);
variant = "land_map";
}
if has_extension(lower, "dat") && (lower.starts_with("units/") || lower.contains("/units/")) {
bump(metrics, "unit_dat_files", 1);
variant = "unit_dat";
}
variant.to_string()
}
fn inspect_nres_metrics(bytes: Vec<u8>, metrics: &mut BTreeMap<String, u64>) -> Result<(), String> {
let entries = inspect_nres_entries(bytes)?;
bump(metrics, "nres_entries", entries.len() as u64);
for entry in entries {
let name = String::from_utf8_lossy(entry.name_bytes()).to_ascii_lowercase();
if has_extension(&name, "msh") {
bump(metrics, "msh_entries", 1);
}
match entry.meta().type_id {
0x3054_414D => {
bump(metrics, "mat0_entries", 1);
}
0x6D78_6554 => {
bump(metrics, "texm_entries", 1);
}
0x4449_5846 => {
bump(metrics, "fxid_entries", 1);
}
0x5241_4557 => {
bump(metrics, "wear_entries", 1);
}
_ => {}
}
}
Ok(())
}
fn bump(metrics: &mut BTreeMap<String, u64>, key: &str, delta: u64) {
if let Some(value) = metrics.get_mut(key) {
*value = value.saturating_add(delta);
}
}
fn has_extension(path: &str, expected: &str) -> bool {
Path::new(path)
.extension()
.is_some_and(|extension| extension.eq_ignore_ascii_case(expected))
}
fn inspect_nres_entries(bytes: Vec<u8>) -> Result<Vec<fparkan_nres::NresEntry>, String> {
let document = fparkan_nres::decode(
Arc::from(bytes.into_boxed_slice()),
fparkan_nres::ReadProfile::Compatible,
)
.map_err(|err| err.to_string())?;
Ok(document.entries().to_vec())
}
/// Computes stable manifest fingerprint.
#[must_use]
pub fn fingerprint(manifest: &CorpusManifest) -> Sha256Digest {
let mut bytes = Vec::new();
for file in &manifest.files {
bytes.extend_from_slice(file.path.as_bytes());
bytes.push(0);
bytes.extend_from_slice(&file.size.to_le_bytes());
bytes.extend_from_slice(&file.hash);
}
sha256(&bytes)
}
/// Writes report atomically.
///
/// # Errors
///
/// Returns [`CorpusError`] if the parent directory, temporary file, write, or
/// final rename operation fails.
pub fn write_report_atomic(path: &Path, report: &CorpusReport) -> Result<(), CorpusError> {
let tmp = path.with_extension("tmp");
if let Some(parent) = path.parent() {
fs::create_dir_all(parent).map_err(|source| CorpusError::Io {
path: parent.to_path_buf(),
source,
})?;
}
let mut file = fs::File::create(&tmp).map_err(|source| CorpusError::Io {
path: tmp.clone(),
source,
})?;
file.write_all(render_report_json(report).as_bytes())
.map_err(|source| CorpusError::Io {
path: tmp.clone(),
source,
})?;
file.sync_all().map_err(|source| CorpusError::Io {
path: tmp.clone(),
source,
})?;
fs::rename(&tmp, path).map_err(|source| CorpusError::Io {
path: path.to_path_buf(),
source,
})?;
Ok(())
}
/// Renders report JSON.
#[must_use]
pub fn render_report_json(report: &CorpusReport) -> String {
let mut out = format!(
"{{\"schema_version\":\"fparkan-corpus-report-v1\",\"schema\":{},\"kind\":\"{:?}\",\"files\":{},\"bytes\":{},\"casefold_collisions\":{},\"fingerprint\":\"{}\",\"failures\":{},\"record_count\":{},\"metrics\":{{",
report.schema,
report.kind,
report.files,
report.bytes,
report.casefold_collisions,
sha256_hex(&report.fingerprint),
report.failures,
report.records.len()
);
for (idx, (key, value)) in report.metrics.iter().enumerate() {
if idx > 0 {
out.push(',');
}
out.push('"');
out.push_str(key);
out.push_str("\":");
out.push_str(&value.to_string());
}
out.push_str("}}");
out.push('}');
out
}
#[cfg(test)]
mod tests {
use super::*;
use fparkan_path::join_under;
use std::time::{SystemTime, UNIX_EPOCH};
#[test]
#[ignore = "requires licensed corpus"]
fn report_for_testdata_roots() {
let root = licensed_root("IS");
let manifest = discover(&root, DiscoverOptions::default()).expect("manifest");
let report = report(&root, &manifest).expect("report");
assert!(report.files > 0);
assert!(report.metrics["nres_files"] > 0);
}
#[test]
#[ignore = "requires licensed corpus"]
fn licensed_part1_manifest_profile_and_counts_match_baseline() {
let root = testdata_root("IS");
let manifest = discover(&root, DiscoverOptions::default()).expect("part 1 manifest");
let report = report(&root, &manifest).expect("report");
assert_eq!(manifest.kind, CorpusKind::Part1);
assert_eq!(report.files, 1_017);
assert_eq!(report.metrics["nres_files"], 120);
assert_eq!(report.metrics["rsli_files"], 2);
assert_eq!(report.metrics["tma_files"], 29);
assert_eq!(report.metrics["land_msh_files"], 33);
assert_eq!(report.metrics["land_map_files"], 33);
assert_eq!(report.metrics["unit_dat_files"], 425);
}
#[test]
#[ignore = "requires licensed corpus"]
fn licensed_part2_manifest_profile_and_counts_match_baseline() {
let root = testdata_root("IS2");
let manifest = discover(&root, DiscoverOptions::default()).expect("part 2 manifest");
let report = report(&root, &manifest).expect("report");
assert_eq!(manifest.kind, CorpusKind::Part2);
assert_eq!(report.files, 1_302);
assert_eq!(report.metrics["nres_files"], 134);
assert_eq!(report.metrics["rsli_files"], 2);
assert_eq!(report.metrics["tma_files"], 31);
assert_eq!(report.metrics["land_msh_files"], 32);
assert_eq!(report.metrics["land_map_files"], 32);
assert_eq!(report.metrics["unit_dat_files"], 676);
}
#[test]
#[ignore = "requires licensed corpus"]
fn licensed_part1_has_no_casefold_relative_path_collisions() {
let root = testdata_root("IS");
let manifest = discover(&root, DiscoverOptions::default()).expect("part 1 manifest");
assert!(manifest.casefold_collisions.is_empty());
}
#[test]
#[ignore = "requires licensed corpus"]
fn licensed_part2_has_no_casefold_relative_path_collisions() {
let root = testdata_root("IS2");
let manifest = discover(&root, DiscoverOptions::default()).expect("part 2 manifest");
assert!(manifest.casefold_collisions.is_empty());
}
#[test]
#[ignore = "requires licensed corpus"]
fn licensed_part1_paths_stay_under_root() {
assert_discovered_paths_stay_under_root("IS");
}
#[test]
#[ignore = "requires licensed corpus"]
fn licensed_part2_paths_stay_under_root() {
assert_discovered_paths_stay_under_root("IS2");
}
#[test]
fn report_json_contains_metrics_and_hashes_not_paths_or_payloads() {
let manifest = CorpusManifest {
kind: CorpusKind::Part1,
files: vec![ManifestEntry {
path: "secret/payload.bin".to_string(),
size: 4,
hash: sha256(b"DATA"),
}],
casefold_collisions: Vec::new(),
};
let report = report(Path::new("."), &manifest).expect("report");
let json = render_report_json(&report);
assert!(json.contains("\"schema_version\":\"fparkan-corpus-report-v1\""));
assert!(json.contains("\"fingerprint\":"));
assert!(json.contains("\"failures\":1"));
assert!(json.contains("\"record_count\":1"));
assert!(json.contains("\"metrics\":"));
assert!(!json.contains("secret/payload.bin"));
assert!(!json.contains("DATA"));
}
#[test]
fn report_records_missing_manifest_files_as_failures() {
let root = temp_dir("report-missing");
let manifest = CorpusManifest {
kind: CorpusKind::Unknown,
files: vec![ManifestEntry {
path: "missing.lib".to_string(),
size: 1,
hash: sha256(b"missing"),
}],
casefold_collisions: Vec::new(),
};
let report = report(&root, &manifest).expect("report");
assert_eq!(report.failures, 1);
assert_eq!(report.records.len(), 1);
assert_eq!(report.records[0].path, "missing.lib");
assert_eq!(report.records[0].status, CorpusFileStatus::Error);
let _ = fs::remove_dir_all(root);
}
#[test]
fn report_records_malformed_nres_as_failure() {
let root = temp_dir("report-bad-nres");
fs::write(root.join("bad.lib"), b"NRes").expect("bad nres");
let manifest = CorpusManifest {
kind: CorpusKind::Unknown,
files: vec![ManifestEntry {
path: "bad.lib".to_string(),
size: 4,
hash: sha256(b"NRes"),
}],
casefold_collisions: Vec::new(),
};
let report = report(&root, &manifest).expect("report");
assert_eq!(report.failures, 1);
assert_eq!(report.records[0].status, CorpusFileStatus::Error);
assert_eq!(report.records[0].variant, "nres");
assert!(report.records[0]
.message
.as_deref()
.is_some_and(|message| message.contains("NRes")));
let _ = fs::remove_dir_all(root);
}
#[test]
fn report_uses_production_nres_parser_for_entry_metrics() {
let root = temp_dir("report-nres");
let archive = build_nres(&[
TestNresEntry {
name: "mesh.msh",
type_id: 0,
payload: b"mesh",
},
TestNresEntry {
name: "mat.bin",
type_id: 0x3054_414D,
payload: b"mat0",
},
TestNresEntry {
name: "texture.bin",
type_id: 0x6D78_6554,
payload: b"texm",
},
]);
fs::write(root.join("archive.lib"), &archive).expect("archive");
let manifest = CorpusManifest {
kind: CorpusKind::Unknown,
files: vec![ManifestEntry {
path: "archive.lib".to_string(),
size: u64::try_from(archive.len()).expect("archive size"),
hash: sha256(&archive),
}],
casefold_collisions: Vec::new(),
};
let report = report(&root, &manifest).expect("report");
assert_eq!(report.failures, 0);
assert_eq!(report.records.len(), 1);
assert_eq!(report.records[0].status, CorpusFileStatus::Ok);
assert_eq!(report.records[0].variant, "nres");
assert_eq!(report.metrics["nres_files"], 1);
assert_eq!(report.metrics["nres_entries"], 3);
assert_eq!(report.metrics["msh_entries"], 1);
assert_eq!(report.metrics["mat0_entries"], 1);
assert_eq!(report.metrics["texm_entries"], 1);
let _ = fs::remove_dir_all(root);
}
#[test]
fn deterministic_traversal_is_creation_order_independent() {
let first = temp_dir("order-first");
let second = temp_dir("order-second");
fs::create_dir_all(first.join("nested")).expect("first nested");
fs::create_dir_all(second.join("nested")).expect("second nested");
fs::write(first.join("b.bin"), b"b").expect("first b");
fs::write(first.join("nested").join("a.bin"), b"a").expect("first a");
fs::write(second.join("nested").join("a.bin"), b"a").expect("second a");
fs::write(second.join("b.bin"), b"b").expect("second b");
let first_manifest = discover(&first, DiscoverOptions::default()).expect("first manifest");
let second_manifest =
discover(&second, DiscoverOptions::default()).expect("second manifest");
assert_eq!(first_manifest.files, second_manifest.files);
let _ = fs::remove_dir_all(first);
let _ = fs::remove_dir_all(second);
}
#[cfg(unix)]
#[test]
fn unreadable_directory_produces_error() {
use std::os::unix::fs::PermissionsExt;
let root = temp_dir("unreadable");
let child = root.join("locked");
fs::create_dir_all(&child).expect("locked dir");
fs::set_permissions(&child, fs::Permissions::from_mode(0o000)).expect("lock dir");
let result = discover(&root, DiscoverOptions::default());
fs::set_permissions(&child, fs::Permissions::from_mode(0o700)).expect("unlock dir");
let _ = fs::remove_dir_all(root);
assert!(matches!(result, Err(CorpusError::Io { path, .. }) if path.ends_with("locked")));
}
#[cfg(unix)]
#[test]
fn symlink_loop_is_not_traversed_by_default() {
use std::os::unix::fs::symlink;
let root = temp_dir("symlink-loop");
fs::write(root.join("real.bin"), b"real").expect("real file");
symlink(&root, root.join("loop")).expect("loop symlink");
let manifest = discover(&root, DiscoverOptions::default()).expect("manifest");
assert_eq!(manifest.files.len(), 1);
assert_eq!(manifest.files[0].path, "real.bin");
let _ = fs::remove_dir_all(root);
}
#[test]
fn casefold_collisions_are_registered() {
let manifest = CorpusManifest {
kind: CorpusKind::Unknown,
files: vec![
ManifestEntry {
path: "Textures/Foo.TEX".to_string(),
size: 1,
hash: sha256(b"first"),
},
ManifestEntry {
path: "textures/foo.tex".to_string(),
size: 1,
hash: sha256(b"second"),
},
],
casefold_collisions: Vec::new(),
};
let collisions = detect_casefold_collisions(&manifest.files);
assert_eq!(
collisions,
vec![vec![
"Textures/Foo.TEX".to_string(),
"textures/foo.tex".to_string()
]]
);
}
#[test]
fn fingerprint_changes() {
let mut manifest = CorpusManifest {
kind: CorpusKind::Unknown,
files: vec![ManifestEntry {
path: "a".to_string(),
size: 1,
hash: sha256(b"before"),
}],
casefold_collisions: Vec::new(),
};
let a = fingerprint(&manifest);
manifest.files[0].hash = sha256(b"after");
assert_ne!(a, fingerprint(&manifest));
}
#[test]
fn atomic_report_write() {
let tmp = std::env::temp_dir().join(format!(
"fparkan-report-{}.json",
SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("clock")
.as_nanos()
));
let report = CorpusReport {
schema: 1,
kind: CorpusKind::Unknown,
files: 0,
bytes: 0,
metrics: BTreeMap::new(),
casefold_collisions: 0,
fingerprint: sha256(b"empty-report"),
records: Vec::new(),
failures: 0,
};
write_report_atomic(&tmp, &report).expect("write");
assert!(tmp.is_file());
let _ = fs::remove_file(tmp);
}
struct TestNresEntry<'a> {
name: &'a str,
type_id: u32,
payload: &'a [u8],
}
fn build_nres(entries: &[TestNresEntry<'_>]) -> Vec<u8> {
let mut out = vec![0; 16];
let mut offsets = Vec::with_capacity(entries.len());
for entry in entries {
offsets.push(u32::try_from(out.len()).expect("offset"));
out.extend_from_slice(entry.payload);
let padding = (8 - (out.len() % 8)) % 8;
out.resize(out.len() + padding, 0);
}
let mut order: Vec<usize> = (0..entries.len()).collect();
order.sort_by(|left, right| {
entries[*left]
.name
.as_bytes()
.cmp(entries[*right].name.as_bytes())
});
for (index, entry) in entries.iter().enumerate() {
push_u32(&mut out, entry.type_id);
push_u32(&mut out, 0);
push_u32(&mut out, 0);
push_u32(
&mut out,
u32::try_from(entry.payload.len()).expect("payload size"),
);
push_u32(&mut out, 0);
let mut name = [0; 36];
let name_bytes = entry.name.as_bytes();
name[..name_bytes.len()].copy_from_slice(name_bytes);
out.extend_from_slice(&name);
push_u32(&mut out, offsets[index]);
push_u32(&mut out, u32::try_from(order[index]).expect("sort index"));
}
out[0..4].copy_from_slice(b"NRes");
out[4..8].copy_from_slice(&0x100_u32.to_le_bytes());
out[8..12].copy_from_slice(&u32::try_from(entries.len()).expect("count").to_le_bytes());
let total_size = u32::try_from(out.len()).expect("total size");
out[12..16].copy_from_slice(&total_size.to_le_bytes());
out
}
fn push_u32(out: &mut Vec<u8>, value: u32) {
out.extend_from_slice(&value.to_le_bytes());
}
fn temp_dir(name: &str) -> PathBuf {
let path = std::env::temp_dir().join(format!(
"fparkan-corpus-{name}-{}",
SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("clock")
.as_nanos()
));
fs::create_dir_all(&path).expect("temp dir");
path
}
fn testdata_root(part: &str) -> PathBuf {
licensed_root(part)
}
fn licensed_root(part: &str) -> PathBuf {
let variable = match part {
"IS" => "FPARKAN_CORPUS_PART1_ROOT",
"IS2" => "FPARKAN_CORPUS_PART2_ROOT",
_ => panic!("unknown licensed corpus part: {part}"),
};
let root = std::env::var_os(variable)
.map(PathBuf::from)
.unwrap_or_else(|| panic!("{variable} is required for licensed corpus tests"));
assert!(
root.is_dir(),
"licensed corpus root is missing: {}",
root.display()
);
root
}
fn assert_discovered_paths_stay_under_root(part: &str) {
let root = testdata_root(part);
let manifest = discover(&root, DiscoverOptions::default()).expect("licensed manifest");
for entry in &manifest.files {
let normalized = normalize_relative(entry.path.as_bytes(), PathPolicy::HostCompatible)
.expect("discovered path should re-normalize");
let joined = join_under(&root, &normalized).expect("discovered path should join");
assert!(
joined.starts_with(&root),
"discovered path escaped root: {}",
entry.path
);
}
}
}
+11
View File
@@ -0,0 +1,11 @@
[package]
name = "fparkan-diagnostics"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
[lints]
workspace = true
+301
View File
@@ -0,0 +1,301 @@
#![forbid(unsafe_code)]
//! Structured diagnostics shared by `FParkan` crates.
/// Diagnostic severity.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum Severity {
/// Informational note.
Info,
/// Recoverable warning.
Warning,
/// Error for the current operation.
Error,
/// Fatal error for the current run.
Fatal,
}
/// Evidence level for a contract or interpretation.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum EvidenceStatus {
/// Described by project documentation.
Documented,
/// Verified by synthetic fixtures.
SyntheticVerified,
/// Verified against the licensed corpus.
CorpusVerified,
/// Verified by runtime capture.
RuntimeCaptured,
/// Working hypothesis; not a runtime contract.
Hypothesis,
}
/// Operation phase where a diagnostic was produced.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum Phase {
/// Discovery.
Discover,
/// Read.
Read,
/// Parse.
Parse,
/// Validate.
Validate,
/// Resolve.
Resolve,
/// Prepare.
Prepare,
/// Construct.
Construct,
/// Register.
Register,
/// Simulate.
Simulate,
/// Render.
Render,
}
/// Byte span in an input source.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct SourceSpan {
/// Start offset.
pub offset: u64,
/// Length in bytes.
pub length: u64,
}
/// Stable diagnostic code.
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
pub struct DiagnosticCode(pub &'static str);
/// Context attached to a diagnostic.
#[derive(Clone, Debug, Default, Eq, PartialEq)]
pub struct DiagnosticContext {
/// Phase.
pub phase: Option<Phase>,
/// Redacted or logical path.
pub path: Option<String>,
/// Archive entry name.
pub archive_entry: Option<String>,
/// Object/prototype key.
pub object_key: Option<String>,
/// Input span.
pub span: Option<SourceSpan>,
}
/// Structured diagnostic with cause chain.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct Diagnostic {
/// Stable code.
pub code: DiagnosticCode,
/// Severity.
pub severity: Severity,
/// Human message.
pub message: String,
/// Context.
pub context: DiagnosticContext,
/// Causes.
pub causes: Vec<Diagnostic>,
}
/// Creates a diagnostic with default error severity.
#[must_use]
pub fn diagnostic(code: DiagnosticCode, message: impl Into<String>) -> Diagnostic {
Diagnostic {
code,
severity: Severity::Error,
message: message.into(),
context: DiagnosticContext::default(),
causes: Vec::new(),
}
}
impl Diagnostic {
/// Returns a copy with severity changed.
#[must_use]
pub fn with_severity(mut self, severity: Severity) -> Self {
self.severity = severity;
self
}
/// Returns a copy with context changed.
#[must_use]
pub fn with_context(mut self, context: DiagnosticContext) -> Self {
self.context = context;
self
}
/// Adds a cause.
pub fn push_cause(&mut self, cause: Diagnostic) {
self.causes.push(cause);
}
}
/// Renders a compact human-readable diagnostic.
#[must_use]
pub fn render_human(diagnostic: &Diagnostic) -> String {
let mut out = format!(
"{:?} {}: {}",
diagnostic.severity, diagnostic.code.0, diagnostic.message
);
if let Some(path) = &diagnostic.context.path {
out.push_str(" [");
out.push_str(path);
out.push(']');
}
out
}
/// Renders deterministic JSON without requiring a serialization dependency.
#[must_use]
pub fn render_json(diagnostic: &Diagnostic) -> String {
fn esc(value: &str) -> String {
let mut out = String::with_capacity(value.len() + 2);
for ch in value.chars() {
match ch {
'\\' => out.push_str("\\\\"),
'"' => out.push_str("\\\""),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
_ => out.push(ch),
}
}
out
}
let mut out = String::new();
out.push('{');
out.push_str("\"code\":\"");
out.push_str(&esc(diagnostic.code.0));
out.push_str("\",\"severity\":\"");
out.push_str(match diagnostic.severity {
Severity::Info => "info",
Severity::Warning => "warning",
Severity::Error => "error",
Severity::Fatal => "fatal",
});
out.push_str("\",\"message\":\"");
out.push_str(&esc(&diagnostic.message));
out.push_str("\",\"context\":{");
if let Some(phase) = diagnostic.context.phase {
out.push_str("\"phase\":\"");
out.push_str(match phase {
Phase::Discover => "discover",
Phase::Read => "read",
Phase::Parse => "parse",
Phase::Validate => "validate",
Phase::Resolve => "resolve",
Phase::Prepare => "prepare",
Phase::Construct => "construct",
Phase::Register => "register",
Phase::Simulate => "simulate",
Phase::Render => "render",
});
out.push('"');
}
if let Some(path) = &diagnostic.context.path {
if diagnostic.context.phase.is_some() {
out.push(',');
}
out.push_str("\"path\":\"");
out.push_str(&esc(path));
out.push('"');
}
if let Some(entry) = &diagnostic.context.archive_entry {
if diagnostic.context.phase.is_some() || diagnostic.context.path.is_some() {
out.push(',');
}
out.push_str("\"archive_entry\":\"");
out.push_str(&esc(entry));
out.push('"');
}
if let Some(key) = &diagnostic.context.object_key {
if diagnostic.context.phase.is_some()
|| diagnostic.context.path.is_some()
|| diagnostic.context.archive_entry.is_some()
{
out.push(',');
}
out.push_str("\"object_key\":\"");
out.push_str(&esc(key));
out.push('"');
}
if let Some(span) = diagnostic.context.span {
if diagnostic.context.phase.is_some()
|| diagnostic.context.path.is_some()
|| diagnostic.context.archive_entry.is_some()
|| diagnostic.context.object_key.is_some()
{
out.push(',');
}
out.push_str("\"span\":{\"offset\":");
out.push_str(&span.offset.to_string());
out.push_str(",\"length\":");
out.push_str(&span.length.to_string());
out.push('}');
}
out.push_str("},\"causes\":[");
for (idx, cause) in diagnostic.causes.iter().enumerate() {
if idx > 0 {
out.push(',');
}
out.push_str(&render_json(cause));
}
out.push_str("]}");
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn json_is_stable() {
let d = diagnostic(DiagnosticCode("S0-DIAG-001"), "keeps context").with_context(
DiagnosticContext {
phase: Some(Phase::Parse),
..DiagnosticContext::default()
},
);
assert_eq!(
render_json(&d),
"{\"code\":\"S0-DIAG-001\",\"severity\":\"error\",\"message\":\"keeps context\",\"context\":{\"phase\":\"parse\"},\"causes\":[]}"
);
}
#[test]
fn diagnostic_chain_preserves_context() {
let mut root = diagnostic(DiagnosticCode("ROOT"), "root").with_context(DiagnosticContext {
phase: Some(Phase::Resolve),
path: Some("archives/material.lib".to_string()),
archive_entry: Some("MATERIAL.MAT0".to_string()),
object_key: Some("unit/tank".to_string()),
span: Some(SourceSpan {
offset: 12,
length: 4,
}),
});
root.push_cause(diagnostic(DiagnosticCode("CAUSE"), "cause").with_context(
DiagnosticContext {
phase: Some(Phase::Parse),
path: Some("archives/material.lib".to_string()),
span: Some(SourceSpan {
offset: 16,
length: 8,
}),
..DiagnosticContext::default()
},
));
let json = render_json(&root);
assert!(json.contains("\"code\":\"ROOT\""));
assert!(json.contains("\"phase\":\"resolve\""));
assert!(json.contains("\"path\":\"archives/material.lib\""));
assert!(json.contains("\"archive_entry\":\"MATERIAL.MAT0\""));
assert!(json.contains("\"object_key\":\"unit/tank\""));
assert!(json.contains("\"span\":{\"offset\":12,\"length\":4}"));
assert!(json.contains("\"code\":\"CAUSE\""));
assert!(json.contains("\"span\":{\"offset\":16,\"length\":8}"));
}
}
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "fparkan-fx"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-binary = { path = "../fparkan-binary" }
[dev-dependencies]
fparkan-nres = { path = "../fparkan-nres" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+18
View File
@@ -0,0 +1,18 @@
[package]
name = "fparkan-material"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
encoding_rs = "0.8"
fparkan-path = { path = "../fparkan-path" }
fparkan-resource = { path = "../fparkan-resource" }
[dev-dependencies]
fparkan-nres = { path = "../fparkan-nres" }
fparkan-vfs = { path = "../fparkan-vfs" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "fparkan-mission-format"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
encoding_rs = "0.8"
fparkan-binary = { path = "../fparkan-binary" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "fparkan-msh"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
encoding_rs = "0.8"
fparkan-nres = { path = "../fparkan-nres" }
[dev-dependencies]
fparkan-animation = { path = "../fparkan-animation" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "fparkan-nres"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-binary = { path = "../fparkan-binary" }
fparkan-path = { path = "../fparkan-path" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+11
View File
@@ -0,0 +1,11 @@
[package]
name = "fparkan-path"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
[lints]
workspace = true
+303
View File
@@ -0,0 +1,303 @@
#![forbid(unsafe_code)]
//! Legacy path normalization and ASCII lookup semantics.
use std::fmt;
use std::path::{Path, PathBuf};
/// Original bytes.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct OriginalPathBytes(pub Vec<u8>);
impl OriginalPathBytes {
/// Returns the preserved byte image.
#[must_use]
pub fn as_bytes(&self) -> &[u8] {
&self.0
}
/// Returns the preserved byte image as an owned vector.
#[must_use]
pub fn into_vec(self) -> Vec<u8> {
self.0
}
}
/// Normalized relative path.
#[derive(Clone, Debug, Eq, Ord, PartialEq, PartialOrd)]
pub struct NormalizedPath(String);
impl NormalizedPath {
/// Returns string view.
#[must_use]
pub fn as_str(&self) -> &str {
&self.0
}
}
/// Normalized path paired with its original byte image.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct NormalizedPathWithOriginal {
normalized: NormalizedPath,
original: OriginalPathBytes,
}
impl NormalizedPathWithOriginal {
/// Returns normalized path.
#[must_use]
pub fn normalized(&self) -> &NormalizedPath {
&self.normalized
}
/// Returns original path bytes.
#[must_use]
pub fn original(&self) -> &OriginalPathBytes {
&self.original
}
/// Splits into normalized and original path parts.
#[must_use]
pub fn into_parts(self) -> (NormalizedPath, OriginalPathBytes) {
(self.normalized, self.original)
}
}
/// ASCII lookup key.
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
pub struct LookupKey(pub Vec<u8>);
/// Resource name bytes.
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
pub struct ResourceName(pub Vec<u8>);
/// Path policy.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum PathPolicy {
/// Strict legacy relative resource path.
StrictLegacy,
/// Host compatible relative path.
HostCompatible,
}
/// Path error.
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum PathError {
/// Empty path.
Empty,
/// Embedded NUL.
EmbeddedNul,
/// Absolute path.
Absolute,
/// Parent traversal.
ParentTraversal,
/// Host path escape.
EscapesRoot,
/// Invalid UTF-8 after normalization.
InvalidUtf8,
}
impl fmt::Display for PathError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Empty => write!(f, "path is empty"),
Self::EmbeddedNul => write!(f, "path contains an embedded NUL byte"),
Self::Absolute => write!(f, "path must be relative and cannot be absolute"),
Self::ParentTraversal => write!(f, "path attempts to traverse outside its root"),
Self::EscapesRoot => write!(f, "normalized path escapes the configured root"),
Self::InvalidUtf8 => write!(f, "path is not valid UTF-8 after normalization"),
}
}
}
impl std::error::Error for PathError {}
/// Normalizes a relative path.
///
/// # Errors
///
/// Returns [`PathError`] when the input is empty, absolute, contains an
/// embedded NUL, attempts parent traversal, or is not valid UTF-8 after
/// legacy separator normalization.
pub fn normalize_relative(raw: &[u8], policy: PathPolicy) -> Result<NormalizedPath, PathError> {
if raw.is_empty() {
return Err(PathError::Empty);
}
if raw.contains(&0) {
return Err(PathError::EmbeddedNul);
}
let text = std::str::from_utf8(raw).map_err(|_| PathError::InvalidUtf8)?;
if text.starts_with('/') || text.starts_with('\\') || has_drive_prefix(text) {
return Err(PathError::Absolute);
}
let mut parts = Vec::new();
for part in text.split(['/', '\\']) {
if part.is_empty() || part == "." {
if policy == PathPolicy::StrictLegacy {
return Err(PathError::ParentTraversal);
}
continue;
}
if part == ".." {
return Err(PathError::ParentTraversal);
}
if policy == PathPolicy::StrictLegacy && part.contains(':') {
return Err(PathError::Absolute);
}
parts.push(part);
}
if parts.is_empty() {
return Err(PathError::Empty);
}
Ok(NormalizedPath(parts.join("/")))
}
/// Normalizes a relative path while preserving its original bytes.
///
/// # Errors
///
/// Returns [`PathError`] under the same conditions as [`normalize_relative`].
pub fn normalize_relative_with_original(
raw: &[u8],
policy: PathPolicy,
) -> Result<NormalizedPathWithOriginal, PathError> {
let normalized = normalize_relative(raw, policy)?;
Ok(NormalizedPathWithOriginal {
normalized,
original: OriginalPathBytes(raw.to_vec()),
})
}
fn has_drive_prefix(text: &str) -> bool {
let bytes = text.as_bytes();
bytes.len() >= 2 && bytes[1] == b':' && bytes[0].is_ascii_alphabetic()
}
/// Builds an ASCII-only casefold lookup key.
#[must_use]
pub fn ascii_lookup_key(raw: &[u8]) -> LookupKey {
LookupKey(raw.iter().map(u8::to_ascii_uppercase).collect())
}
/// Ensures relative path does not escape.
///
/// # Errors
///
/// Returns [`PathError::ParentTraversal`] when a normalized segment attempts
/// to address a parent directory.
pub fn reject_escape(rel: &NormalizedPath) -> Result<(), PathError> {
if rel.0.split('/').any(|part| part == "..") {
Err(PathError::ParentTraversal)
} else {
Ok(())
}
}
/// Joins normalized path under root.
///
/// # Errors
///
/// Returns [`PathError`] if the normalized path fails the escape check.
pub fn join_under(root: &Path, rel: &NormalizedPath) -> Result<PathBuf, PathError> {
reject_escape(rel)?;
Ok(root.join(rel.as_str()))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn normalizes_separators() {
let p = normalize_relative(b"DATA\\MAPS/INTRO/Land.msh", PathPolicy::StrictLegacy)
.expect("path");
assert_eq!(p.as_str(), "DATA/MAPS/INTRO/Land.msh");
}
#[test]
fn rejects_escape() {
assert_eq!(
normalize_relative(b"DATA/../secret", PathPolicy::StrictLegacy),
Err(PathError::ParentTraversal)
);
}
#[test]
fn rejects_absolute_drive_and_nul_paths() {
assert_eq!(
normalize_relative(b"/DATA/MAPS", PathPolicy::StrictLegacy),
Err(PathError::Absolute)
);
assert_eq!(
normalize_relative(b"C:\\DATA\\MAPS", PathPolicy::StrictLegacy),
Err(PathError::Absolute)
);
assert_eq!(
normalize_relative(b"DATA\0MAPS", PathPolicy::StrictLegacy),
Err(PathError::EmbeddedNul)
);
}
#[test]
fn path_error_display_is_actionable() {
assert_eq!(
PathError::ParentTraversal.to_string(),
"path attempts to traverse outside its root"
);
assert_eq!(
PathError::EmbeddedNul.to_string(),
"path contains an embedded NUL byte"
);
}
#[test]
fn strict_legacy_rejects_host_only_segments() {
assert_eq!(
normalize_relative(b"./DATA/MAPS", PathPolicy::StrictLegacy),
Err(PathError::ParentTraversal)
);
assert_eq!(
normalize_relative(b"DATA//MAPS", PathPolicy::StrictLegacy),
Err(PathError::ParentTraversal)
);
assert_eq!(
normalize_relative(b"DATA/stream:name", PathPolicy::StrictLegacy),
Err(PathError::Absolute)
);
let host = normalize_relative(b"./DATA//MAPS", PathPolicy::HostCompatible).expect("host");
assert_eq!(host.as_str(), "DATA/MAPS");
}
#[test]
fn join_under_keeps_normalized_path_below_root() {
let rel = normalize_relative(b"DATA/MAPS/Land.map", PathPolicy::StrictLegacy)
.expect("relative path");
let joined = join_under(Path::new("/game"), &rel).expect("join");
assert_eq!(joined, PathBuf::from("/game/DATA/MAPS/Land.map"));
}
#[test]
fn ascii_casefold_does_not_unicode_fold() {
assert_eq!(ascii_lookup_key(b"AbZ\xD0"), LookupKey(b"ABZ\xD0".to_vec()));
}
#[test]
fn non_ascii_original_bytes_remain_stable() {
let raw = "DATA/Тест.bin".as_bytes();
let path = normalize_relative_with_original(raw, PathPolicy::StrictLegacy)
.expect("path with non-ASCII UTF-8");
assert_eq!(path.normalized().as_str().as_bytes(), raw);
assert_eq!(path.original().as_bytes(), raw);
assert_eq!(&ascii_lookup_key(raw).0[5..13], &raw[5..13]);
}
#[test]
fn original_separators_and_raw_bytes_are_preserved() {
let raw = b"DATA\\Maps/Intro\\Land.msh";
let path = normalize_relative_with_original(raw, PathPolicy::StrictLegacy).expect("path");
assert_eq!(path.normalized().as_str(), "DATA/Maps/Intro/Land.msh");
assert_eq!(path.original().as_bytes(), raw);
}
}
+11
View File
@@ -0,0 +1,11 @@
[package]
name = "fparkan-platform"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
[lints]
workspace = true
+93
View File
@@ -0,0 +1,93 @@
#![forbid(unsafe_code)]
//! Platform ports for clocks, input, events, windows, and graphics requests.
/// Monotonic instant.
#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd)]
pub struct MonotonicInstant(pub u64);
/// Monotonic clock.
pub trait MonotonicClock {
/// Current instant.
fn now(&self) -> MonotonicInstant;
}
/// Platform event.
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum PlatformEvent {
/// Quit requested.
Quit,
}
/// Platform error.
#[derive(Debug)]
pub enum PlatformError {
/// Backend failed.
Backend,
}
impl std::fmt::Display for PlatformError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{self:?}")
}
}
impl std::error::Error for PlatformError {}
/// Event source.
pub trait EventSource {
/// Polls events.
///
/// # Errors
///
/// Returns [`PlatformError`] when the backend cannot collect events.
fn poll(&mut self, out: &mut Vec<PlatformEvent>) -> Result<(), PlatformError>;
}
/// Physical size.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct PhysicalSize {
/// Width.
pub width: u32,
/// Height.
pub height: u32,
}
/// Window port.
pub trait WindowPort {
/// Drawable size.
fn drawable_size(&self) -> PhysicalSize;
/// Presents.
///
/// # Errors
///
/// Returns [`PlatformError`] when the backend cannot present the current
/// frame.
fn present(&mut self) -> Result<(), PlatformError>;
}
/// Graphics profile.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum GraphicsProfile {
/// Desktop core.
DesktopCore,
/// Embedded profile.
Embedded,
}
/// Version.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct Version {
/// Major.
pub major: u8,
/// Minor.
pub minor: u8,
}
/// Graphics context request.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct GraphicsContextRequest {
/// Profile.
pub profile: GraphicsProfile,
/// Version.
pub version: Version,
}
+20
View File
@@ -0,0 +1,20 @@
[package]
name = "fparkan-prototype"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
encoding_rs = "0.8"
fparkan-binary = { path = "../fparkan-binary" }
fparkan-material = { path = "../fparkan-material" }
fparkan-msh = { path = "../fparkan-msh" }
fparkan-nres = { path = "../fparkan-nres" }
fparkan-path = { path = "../fparkan-path" }
fparkan-resource = { path = "../fparkan-resource" }
fparkan-texm = { path = "../fparkan-texm" }
fparkan-vfs = { path = "../fparkan-vfs" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+12
View File
@@ -0,0 +1,12 @@
[package]
name = "fparkan-render"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-world = { path = "../fparkan-world" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "fparkan-resource"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-binary = { path = "../fparkan-binary" }
fparkan-nres = { path = "../fparkan-nres" }
fparkan-path = { path = "../fparkan-path" }
fparkan-rsli = { path = "../fparkan-rsli" }
fparkan-vfs = { path = "../fparkan-vfs" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+12
View File
@@ -0,0 +1,12 @@
[package]
name = "fparkan-rsli"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
flate2 = { version = "1", default-features = false, features = ["rust_backend"] }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "fparkan-runtime"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-mission-format = { path = "../fparkan-mission-format" }
fparkan-nres = { path = "../fparkan-nres" }
fparkan-path = { path = "../fparkan-path" }
fparkan-platform = { path = "../fparkan-platform" }
fparkan-prototype = { path = "../fparkan-prototype" }
fparkan-render = { path = "../fparkan-render" }
fparkan-resource = { path = "../fparkan-resource" }
fparkan-terrain = { path = "../fparkan-terrain" }
fparkan-terrain-format = { path = "../fparkan-terrain-format" }
fparkan-vfs = { path = "../fparkan-vfs" }
fparkan-world = { path = "../fparkan-world" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "fparkan-terrain-format"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-binary = { path = "../fparkan-binary" }
fparkan-nres = { path = "../fparkan-nres" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "fparkan-terrain"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-terrain-format = { path = "../fparkan-terrain-format" }
[dev-dependencies]
fparkan-nres = { path = "../fparkan-nres" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+12
View File
@@ -0,0 +1,12 @@
[package]
name = "fparkan-test-support"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-render = { path = "../fparkan-render" }
[lints]
workspace = true
+25
View File
@@ -0,0 +1,25 @@
#![forbid(unsafe_code)]
//! Dev-only synthetic builders and fake ports.
use fparkan_render::{FrameOutput, RenderBackend, RenderCommandList, RenderError};
/// Fake clock.
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
pub struct FakeClock {
/// Current tick.
pub tick: u64,
}
/// Recording backend.
#[derive(Clone, Debug, Default)]
pub struct RecordingRenderBackend {
/// Recorded command lists.
pub captures: Vec<RenderCommandList>,
}
impl RenderBackend for RecordingRenderBackend {
fn execute(&mut self, commands: &RenderCommandList) -> Result<FrameOutput, RenderError> {
self.captures.push(commands.clone());
Ok(FrameOutput)
}
}
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "fparkan-texm"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
[dev-dependencies]
fparkan-nres = { path = "../fparkan-nres" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "fparkan-vfs"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-binary = { path = "../fparkan-binary" }
fparkan-path = { path = "../fparkan-path" }
[lints]
workspace = true
+615
View File
@@ -0,0 +1,615 @@
#![forbid(unsafe_code)]
//! Virtual filesystem ports for resource loading.
use fparkan_binary::{sha256, Sha256Digest};
use fparkan_path::{ascii_lookup_key, join_under, NormalizedPath};
use std::collections::BTreeMap;
use std::fs;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use std::time::SystemTime;
/// VFS metadata.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct VfsMetadata {
/// Byte length.
pub len: u64,
/// SHA-256 content fingerprint for cache invalidation.
pub fingerprint: Sha256Digest,
}
/// VFS entry.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct VfsEntry {
/// Path.
pub path: NormalizedPath,
/// Metadata.
pub metadata: VfsMetadata,
}
/// VFS error.
#[derive(Debug)]
pub enum VfsError {
/// Missing entry.
NotFound(String),
/// Ambiguous host path.
Ambiguous(String),
/// I/O error.
Io(std::io::Error),
/// Invalid path.
Path,
}
impl std::fmt::Display for VfsError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::NotFound(path) => write!(f, "not found: {path}"),
Self::Ambiguous(path) => write!(f, "ambiguous host path: {path}"),
Self::Io(err) => write!(f, "{err}"),
Self::Path => write!(f, "invalid path"),
}
}
}
impl std::error::Error for VfsError {}
/// Resource VFS.
pub trait Vfs: Send + Sync {
/// Reads metadata.
///
/// # Errors
///
/// Returns [`VfsError`] when the path is invalid, missing, or cannot be
/// inspected by the backing store.
fn metadata(&self, path: &NormalizedPath) -> Result<VfsMetadata, VfsError>;
/// Reads bytes.
///
/// # Errors
///
/// Returns [`VfsError`] when the path is invalid, missing, or cannot be
/// read by the backing store.
fn read(&self, path: &NormalizedPath) -> Result<Arc<[u8]>, VfsError>;
/// Lists entries below prefix.
///
/// # Errors
///
/// Returns [`VfsError`] when the prefix is invalid, missing, or cannot be
/// traversed by the backing store.
fn list(&self, prefix: &NormalizedPath) -> Result<Vec<VfsEntry>, VfsError>;
}
/// Host directory VFS.
#[derive(Clone, Debug)]
pub struct DirectoryVfs {
root: PathBuf,
fingerprint_cache: Arc<Mutex<BTreeMap<PathBuf, CachedHostFingerprint>>>,
}
impl DirectoryVfs {
/// Creates a directory VFS.
#[must_use]
pub fn new(root: impl AsRef<Path>) -> Self {
Self {
root: root.as_ref().to_path_buf(),
fingerprint_cache: Arc::default(),
}
}
fn host_path(&self, path: &NormalizedPath) -> Result<PathBuf, VfsError> {
join_under(&self.root, path).map_err(|_| VfsError::Path)?;
resolve_casefolded(&self.root, path.as_str())
}
fn metadata_from_host_file(&self, path: &Path) -> Result<VfsMetadata, VfsError> {
let metadata = fs::symlink_metadata(path).map_err(VfsError::Io)?;
metadata_from_host_file_with_cache(path, &metadata, &self.fingerprint_cache)
}
}
#[derive(Clone, Debug, Eq, PartialEq)]
struct CachedHostFingerprint {
len: u64,
modified: Option<SystemTime>,
fingerprint: Sha256Digest,
}
impl Vfs for DirectoryVfs {
fn metadata(&self, path: &NormalizedPath) -> Result<VfsMetadata, VfsError> {
self.metadata_from_host_file(&self.host_path(path)?)
}
fn read(&self, path: &NormalizedPath) -> Result<Arc<[u8]>, VfsError> {
let host = self.host_path(path)?;
if fs::symlink_metadata(&host)
.map_err(VfsError::Io)?
.file_type()
.is_symlink()
{
return Err(VfsError::Path);
}
let bytes = fs::read(host).map_err(VfsError::Io)?;
Ok(Arc::from(bytes.into_boxed_slice()))
}
fn list(&self, prefix: &NormalizedPath) -> Result<Vec<VfsEntry>, VfsError> {
let base = self.host_path(prefix)?;
let mut entries = Vec::new();
if base.is_file() {
let metadata = fs::symlink_metadata(&base).map_err(VfsError::Io)?;
entries.push(VfsEntry {
path: prefix.clone(),
metadata: metadata_from_host_file_with_cache(
&base,
&metadata,
&self.fingerprint_cache,
)?,
});
return Ok(entries);
}
list_recursive(&self.root, &base, &self.fingerprint_cache, &mut entries)?;
entries.sort_by(|a, b| a.path.as_str().cmp(b.path.as_str()));
Ok(entries)
}
}
fn resolve_casefolded(root: &Path, normalized: &str) -> Result<PathBuf, VfsError> {
let mut current = root.to_path_buf();
for segment in normalized.split('/') {
let read_dir = fs::read_dir(&current).map_err(VfsError::Io)?;
let mut matches = Vec::new();
for entry in read_dir {
let entry = entry.map_err(VfsError::Io)?;
let name = entry.file_name();
let Some(name) = name.to_str() else {
continue;
};
if name.eq_ignore_ascii_case(segment) {
if entry.file_type().map_err(VfsError::Io)?.is_symlink() {
return Err(VfsError::Path);
}
matches.push(entry.path());
}
}
current = select_casefolded_match(normalized, &current, segment, matches)?;
}
Ok(current)
}
fn select_casefolded_match(
normalized: &str,
current: &Path,
segment: &str,
mut matches: Vec<PathBuf>,
) -> Result<PathBuf, VfsError> {
matches.sort();
match matches.len() {
0 => Err(VfsError::NotFound(normalized.to_string())),
1 => Ok(matches.remove(0)),
_ => Err(VfsError::Ambiguous(format!(
"{}/{}",
current.display(),
segment
))),
}
}
fn list_recursive(
root: &Path,
dir: &Path,
fingerprint_cache: &Mutex<BTreeMap<PathBuf, CachedHostFingerprint>>,
out: &mut Vec<VfsEntry>,
) -> Result<(), VfsError> {
let read_dir = fs::read_dir(dir).map_err(VfsError::Io)?;
let mut children = Vec::new();
for entry in read_dir {
let entry = entry.map_err(VfsError::Io)?;
children.push(entry.path());
}
children.sort();
for child in children {
let metadata = fs::symlink_metadata(&child).map_err(VfsError::Io)?;
if metadata.file_type().is_symlink() {
return Err(VfsError::Path);
}
if metadata.is_dir() {
list_recursive(root, &child, fingerprint_cache, out)?;
continue;
}
if !metadata.is_file() {
continue;
}
let rel = child.strip_prefix(root).map_err(|_| VfsError::Path)?;
let rel_text = rel.to_str().ok_or(VfsError::Path)?;
let path = fparkan_path::normalize_relative(
rel_text.as_bytes(),
fparkan_path::PathPolicy::HostCompatible,
)
.map_err(|_| VfsError::Path)?;
out.push(VfsEntry {
path,
metadata: metadata_from_host_file_with_cache(&child, &metadata, fingerprint_cache)?,
});
}
Ok(())
}
fn metadata_from_host_file_with_cache(
path: &Path,
metadata: &fs::Metadata,
fingerprint_cache: &Mutex<BTreeMap<PathBuf, CachedHostFingerprint>>,
) -> Result<VfsMetadata, VfsError> {
if !metadata.is_file() {
return Err(VfsError::Path);
}
let len = metadata.len();
let modified = metadata.modified().ok();
if let Some(cached) = fingerprint_cache
.lock()
.map_err(|_| VfsError::Path)?
.get(path)
.cloned()
.filter(|cached| cached.len == len && cached.modified == modified)
{
return Ok(VfsMetadata {
len,
fingerprint: cached.fingerprint,
});
}
let bytes = fs::read(path).map_err(VfsError::Io)?;
let fingerprint = sha256(&bytes);
fingerprint_cache
.lock()
.map_err(|_| VfsError::Path)?
.insert(
path.to_path_buf(),
CachedHostFingerprint {
len,
modified,
fingerprint,
},
);
Ok(VfsMetadata { len, fingerprint })
}
/// In-memory VFS.
#[derive(Clone, Debug, Default)]
pub struct MemoryVfs {
files: BTreeMap<String, Arc<[u8]>>,
lookup: BTreeMap<Vec<u8>, Vec<String>>,
}
impl MemoryVfs {
/// Inserts a file.
#[allow(clippy::needless_pass_by_value)]
pub fn insert(&mut self, path: NormalizedPath, bytes: Arc<[u8]>) {
let path = path.as_str().to_string();
self.files.insert(path, bytes);
self.rebuild_lookup();
}
fn rebuild_lookup(&mut self) {
self.lookup.clear();
for path in self.files.keys() {
self.lookup
.entry(ascii_lookup_key(path.as_bytes()).0)
.or_default()
.push(path.clone());
}
for paths in self.lookup.values_mut() {
paths.sort();
}
}
fn resolve_path(&self, path: &NormalizedPath) -> Result<&str, VfsError> {
let key = ascii_lookup_key(path.as_str().as_bytes()).0;
let matches = self
.lookup
.get(&key)
.ok_or_else(|| VfsError::NotFound(path.as_str().to_string()))?;
match matches.as_slice() {
[single] => Ok(single.as_str()),
[] => Err(VfsError::NotFound(path.as_str().to_string())),
_ => Err(VfsError::Ambiguous(path.as_str().to_string())),
}
}
}
impl Vfs for MemoryVfs {
fn metadata(&self, path: &NormalizedPath) -> Result<VfsMetadata, VfsError> {
let resolved = self.resolve_path(path)?;
let bytes = self
.files
.get(resolved)
.ok_or_else(|| VfsError::NotFound(path.as_str().to_string()))?;
Ok(VfsMetadata {
len: bytes.len() as u64,
fingerprint: sha256(bytes),
})
}
fn read(&self, path: &NormalizedPath) -> Result<Arc<[u8]>, VfsError> {
let resolved = self.resolve_path(path)?;
self.files
.get(resolved)
.cloned()
.ok_or_else(|| VfsError::NotFound(path.as_str().to_string()))
}
fn list(&self, prefix: &NormalizedPath) -> Result<Vec<VfsEntry>, VfsError> {
let mut out = Vec::new();
for (path, bytes) in &self.files {
if path
.as_bytes()
.get(..prefix.as_str().len())
.is_some_and(|head| head.eq_ignore_ascii_case(prefix.as_str().as_bytes()))
{
let normalized = fparkan_path::normalize_relative(
path.as_bytes(),
fparkan_path::PathPolicy::StrictLegacy,
)
.map_err(|_| VfsError::Path)?;
out.push(VfsEntry {
path: normalized,
metadata: VfsMetadata {
len: bytes.len() as u64,
fingerprint: sha256(bytes),
},
});
}
}
Ok(out)
}
}
/// Layered VFS with deterministic first-layer precedence.
#[derive(Clone, Default)]
pub struct OverlayVfs {
layers: Vec<Arc<dyn Vfs>>,
}
impl std::fmt::Debug for OverlayVfs {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("OverlayVfs")
.field("layers", &self.layers.len())
.finish()
}
}
impl OverlayVfs {
/// Creates an empty overlay.
#[must_use]
pub fn new() -> Self {
Self::default()
}
/// Creates an overlay from ordered layers.
#[must_use]
pub fn from_layers(layers: Vec<Arc<dyn Vfs>>) -> Self {
Self { layers }
}
/// Appends a lower-priority layer.
pub fn push_layer(&mut self, layer: Arc<dyn Vfs>) {
self.layers.push(layer);
}
}
impl Vfs for OverlayVfs {
fn metadata(&self, path: &NormalizedPath) -> Result<VfsMetadata, VfsError> {
for layer in &self.layers {
match layer.metadata(path) {
Ok(metadata) => return Ok(metadata),
Err(VfsError::NotFound(_)) => {}
Err(err) => return Err(err),
}
}
Err(VfsError::NotFound(path.as_str().to_string()))
}
fn read(&self, path: &NormalizedPath) -> Result<Arc<[u8]>, VfsError> {
for layer in &self.layers {
match layer.read(path) {
Ok(bytes) => return Ok(bytes),
Err(VfsError::NotFound(_)) => {}
Err(err) => return Err(err),
}
}
Err(VfsError::NotFound(path.as_str().to_string()))
}
fn list(&self, prefix: &NormalizedPath) -> Result<Vec<VfsEntry>, VfsError> {
let mut by_key = BTreeMap::new();
for layer in &self.layers {
match layer.list(prefix) {
Ok(entries) => {
for entry in entries {
let key = entry.path.as_str().to_ascii_uppercase();
by_key.entry(key).or_insert(entry);
}
}
Err(VfsError::NotFound(_)) => {}
Err(err) => return Err(err),
}
}
let mut entries: Vec<_> = by_key.into_values().collect();
entries.sort_by(|a, b| a.path.as_str().cmp(b.path.as_str()));
Ok(entries)
}
}
#[cfg(test)]
mod tests {
use super::*;
use fparkan_path::{normalize_relative, PathPolicy};
#[test]
fn directory_vfs_resolves_ascii_casefolded_segments() {
let root = unique_test_dir("casefold");
let dir = root.join("data").join("MAPS").join("Tut_1");
std::fs::create_dir_all(&dir).expect("mkdir");
std::fs::write(dir.join("Land.msh"), b"mesh").expect("write");
let vfs = DirectoryVfs::new(&root);
let path = normalize_relative(b"DATA/maps/tut_1/land.MSH", PathPolicy::StrictLegacy)
.expect("path");
assert_eq!(vfs.read(&path).expect("read").as_ref(), b"mesh");
std::fs::remove_dir_all(root).expect("cleanup");
}
#[test]
fn directory_vfs_reports_casefold_ambiguity_even_for_exact_host_path() {
let root = unique_test_dir("casefold-ambiguous");
std::fs::create_dir_all(root.join("Data")).expect("mkdir first");
std::fs::create_dir_all(root.join("data")).expect("mkdir second");
std::fs::write(root.join("Data").join("File.bin"), b"first").expect("write first");
std::fs::write(root.join("data").join("File.bin"), b"second").expect("write second");
let collision_count = std::fs::read_dir(&root)
.expect("read root")
.flatten()
.filter(|entry| {
entry
.file_name()
.to_str()
.is_some_and(|name| name.eq_ignore_ascii_case("data"))
})
.count();
if collision_count < 2 {
std::fs::remove_dir_all(root).expect("cleanup");
return;
}
let vfs = DirectoryVfs::new(&root);
let path = normalize_relative(b"Data/File.bin", PathPolicy::StrictLegacy).expect("path");
assert!(matches!(vfs.read(&path), Err(VfsError::Ambiguous(_))));
std::fs::remove_dir_all(root).expect("cleanup");
}
#[test]
fn directory_vfs_lists_files_below_prefix() {
let root = unique_test_dir("list");
std::fs::create_dir_all(root.join("DATA").join("MAPS")).expect("mkdir");
std::fs::write(root.join("DATA").join("MAPS").join("Land.map"), b"map").expect("write");
std::fs::write(root.join("BuildDat.lst"), b"build").expect("write");
let vfs = DirectoryVfs::new(&root);
let prefix = normalize_relative(b"data", PathPolicy::StrictLegacy).expect("prefix");
let entries = vfs.list(&prefix).expect("list");
assert_eq!(entries.len(), 1);
assert!(entries[0]
.path
.as_str()
.eq_ignore_ascii_case("DATA/MAPS/Land.map"));
std::fs::remove_dir_all(root).expect("cleanup");
}
#[test]
fn directory_vfs_fingerprint_changes_for_same_length_content() {
let root = unique_test_dir("content-fingerprint");
std::fs::create_dir_all(root.join("DATA")).expect("mkdir");
std::fs::write(root.join("DATA").join("File.bin"), b"before").expect("write before");
let vfs = DirectoryVfs::new(&root);
let path = normalize_relative(b"DATA/File.bin", PathPolicy::StrictLegacy).expect("path");
let before = vfs.metadata(&path).expect("before metadata");
std::fs::write(root.join("DATA").join("File.bin"), b"after!").expect("write after");
let after = vfs.metadata(&path).expect("after metadata");
assert_eq!(before.len, after.len);
assert_ne!(before.fingerprint, after.fingerprint);
std::fs::remove_dir_all(root).expect("cleanup");
}
#[cfg(unix)]
#[test]
fn directory_vfs_rejects_symlink_escape() {
let root = unique_test_dir("symlink-escape");
let outside = unique_test_dir("symlink-outside");
std::fs::create_dir_all(&root).expect("mkdir root");
std::fs::create_dir_all(&outside).expect("mkdir outside");
std::fs::write(outside.join("secret.bin"), b"secret").expect("write outside");
std::os::unix::fs::symlink(&outside, root.join("DATA")).expect("symlink");
let vfs = DirectoryVfs::new(&root);
let path = normalize_relative(b"DATA/secret.bin", PathPolicy::StrictLegacy).expect("path");
let prefix = normalize_relative(b"DATA", PathPolicy::StrictLegacy).expect("prefix");
assert!(matches!(vfs.read(&path), Err(VfsError::Path)));
assert!(matches!(vfs.list(&prefix), Err(VfsError::Path)));
std::fs::remove_dir_all(root).expect("cleanup root");
std::fs::remove_dir_all(outside).expect("cleanup outside");
}
#[test]
fn casefold_selector_reports_ambiguous_segments() {
let err = select_casefolded_match(
"data/file.bin",
Path::new("/game"),
"data",
vec![PathBuf::from("/game/Data"), PathBuf::from("/game/DATA")],
)
.expect_err("ambiguous path");
assert!(matches!(err, VfsError::Ambiguous(_)));
}
#[test]
fn memory_vfs_uses_ascii_casefold_lookup() {
let path = normalize_relative(b"Data/File.bin", PathPolicy::StrictLegacy).expect("path");
let mut vfs = MemoryVfs::default();
vfs.insert(path.clone(), Arc::from(b"payload".as_slice()));
assert_eq!(vfs.metadata(&path).expect("metadata").len, 7);
assert_eq!(vfs.read(&path).expect("read").as_ref(), b"payload");
let other_case =
normalize_relative(b"data/file.bin", PathPolicy::StrictLegacy).expect("path");
assert_eq!(
vfs.read(&other_case).expect("casefold read").as_ref(),
b"payload"
);
}
#[test]
fn memory_vfs_reports_casefold_ambiguity() {
let first = normalize_relative(b"Data/File.bin", PathPolicy::StrictLegacy).expect("first");
let second =
normalize_relative(b"DATA/file.BIN", PathPolicy::StrictLegacy).expect("second");
let query = normalize_relative(b"data/file.bin", PathPolicy::StrictLegacy).expect("query");
let mut vfs = MemoryVfs::default();
vfs.insert(first, Arc::from(b"first".as_slice()));
vfs.insert(second, Arc::from(b"second".as_slice()));
assert!(matches!(vfs.read(&query), Err(VfsError::Ambiguous(_))));
}
#[test]
fn overlay_vfs_uses_first_matching_layer() {
let path = normalize_relative(b"DATA/File.bin", PathPolicy::StrictLegacy).expect("path");
let prefix = normalize_relative(b"DATA", PathPolicy::StrictLegacy).expect("prefix");
let mut high = MemoryVfs::default();
let mut low = MemoryVfs::default();
high.insert(path.clone(), Arc::from(b"high".as_slice()));
low.insert(path.clone(), Arc::from(b"low".as_slice()));
let overlay = OverlayVfs::from_layers(vec![Arc::new(high), Arc::new(low)]);
assert_eq!(overlay.read(&path).expect("read").as_ref(), b"high");
let entries = overlay.list(&prefix).expect("list");
assert_eq!(entries.len(), 1);
assert_eq!(entries[0].metadata.len, 4);
}
fn unique_test_dir(name: &str) -> PathBuf {
let mut path = std::env::temp_dir();
path.push(format!("fparkan-vfs-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&path);
path
}
}
+12
View File
@@ -0,0 +1,12 @@
[package]
name = "fparkan-world"
version.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
[dependencies]
fparkan-binary = { path = "../fparkan-binary" }
[lints]
workspace = true
File diff suppressed because it is too large Load Diff
-10
View File
@@ -1,10 +0,0 @@
[package]
name = "nres"
version = "0.1.0"
edition = "2021"
[dependencies]
common = { path = "../common" }
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.61", features = ["Win32_Storage_FileSystem"] }
-42
View File
@@ -1,42 +0,0 @@
# nres
Rust-библиотека для работы с архивами формата **NRes**.
## Что умеет
- Открытие архива из файла (`open_path`) и из памяти (`open_bytes`).
- Поддержка `raw_mode` (весь файл как единый ресурс).
- Чтение метаданных и итерация по записям.
- Поиск по имени без учёта регистра (`find`).
- Чтение данных ресурса (`read`, `read_into`, `raw_slice`).
- Редактирование архива через `Editor`:
- `add`, `replace_data`, `remove`.
- `commit` с пересчётом `sort_index`, выравниванием по 8 байт и атомарной записью файла.
## Модель ошибок
Библиотека возвращает типизированные ошибки (`InvalidMagic`, `UnsupportedVersion`, `TotalSizeMismatch`, `DirectoryOutOfBounds`, `EntryDataOutOfBounds`, и др.) без паник в production-коде.
## Покрытие тестами
### Реальные файлы
- Рекурсивный прогон по `testdata/nres/**`.
- Сейчас в наборе: **120 архивов**.
- Для каждого архива проверяется:
- чтение всех записей;
- `read`/`read_into`/`raw_slice`;
- `find`;
- `unpack -> repack (Editor::commit)` с проверкой **byte-to-byte**.
### Синтетические тесты
- Проверка основных сценариев редактирования (`add/replace/remove/commit`).
- Проверка валидации и ошибок:
- `InvalidMagic`, `UnsupportedVersion`, `TotalSizeMismatch`, `InvalidEntryCount`, `DirectoryOutOfBounds`, `NameTooLong`, `EntryDataOutOfBounds`, `EntryIdOutOfRange`, `NameContainsNul`.
## Быстрый запуск тестов
```bash
cargo test -p nres -- --nocapture
```
-110
View File
@@ -1,110 +0,0 @@
use core::fmt;
#[derive(Debug)]
#[non_exhaustive]
pub enum Error {
Io(std::io::Error),
InvalidMagic {
got: [u8; 4],
},
UnsupportedVersion {
got: u32,
},
TotalSizeMismatch {
header: u32,
actual: u64,
},
InvalidEntryCount {
got: i32,
},
TooManyEntries {
got: usize,
},
DirectoryOutOfBounds {
directory_offset: u64,
directory_len: u64,
file_len: u64,
},
EntryIdOutOfRange {
id: u32,
entry_count: u32,
},
EntryDataOutOfBounds {
id: u32,
offset: u64,
size: u32,
directory_offset: u64,
},
NameTooLong {
got: usize,
max: usize,
},
NameContainsNul,
BadNameEncoding,
IntegerOverflow,
RawModeDisallowsOperation(&'static str),
}
impl From<std::io::Error> for Error {
fn from(value: std::io::Error) -> Self {
Self::Io(value)
}
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Error::Io(e) => write!(f, "I/O error: {e}"),
Error::InvalidMagic { got } => write!(f, "invalid NRes magic: {got:02X?}"),
Error::UnsupportedVersion { got } => {
write!(f, "unsupported NRes version: {got:#x}")
}
Error::TotalSizeMismatch { header, actual } => {
write!(f, "NRes total_size mismatch: header={header}, actual={actual}")
}
Error::InvalidEntryCount { got } => write!(f, "invalid entry_count: {got}"),
Error::TooManyEntries { got } => write!(f, "too many entries: {got} exceeds u32::MAX"),
Error::DirectoryOutOfBounds {
directory_offset,
directory_len,
file_len,
} => write!(
f,
"directory out of bounds: off={directory_offset}, len={directory_len}, file={file_len}"
),
Error::EntryIdOutOfRange { id, entry_count } => {
write!(f, "entry id out of range: id={id}, count={entry_count}")
}
Error::EntryDataOutOfBounds {
id,
offset,
size,
directory_offset,
} => write!(
f,
"entry data out of bounds: id={id}, off={offset}, size={size}, dir_off={directory_offset}"
),
Error::NameTooLong { got, max } => write!(f, "name too long: {got} > {max}"),
Error::NameContainsNul => write!(f, "name contains NUL byte"),
Error::BadNameEncoding => write!(f, "bad name encoding"),
Error::IntegerOverflow => write!(f, "integer overflow"),
Error::RawModeDisallowsOperation(op) => {
write!(f, "operation not allowed in raw mode: {op}")
}
}
}
}
impl std::error::Error for Error {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
Self::Io(err) => Some(err),
_ => None,
}
}
}
-702
View File
@@ -1,702 +0,0 @@
pub mod error;
use crate::error::Error;
use common::{OutputBuffer, ResourceData};
use core::ops::Range;
use std::cmp::Ordering;
use std::fs::{self, OpenOptions as FsOpenOptions};
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::{SystemTime, UNIX_EPOCH};
pub type Result<T> = core::result::Result<T, Error>;
#[derive(Clone, Debug, Default)]
pub struct OpenOptions {
pub raw_mode: bool,
pub sequential_hint: bool,
pub prefetch_pages: bool,
}
#[derive(Clone, Debug, Default)]
pub enum OpenMode {
#[default]
ReadOnly,
ReadWrite,
}
#[derive(Debug)]
pub struct Archive {
bytes: Arc<[u8]>,
entries: Vec<EntryRecord>,
raw_mode: bool,
}
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)]
pub struct EntryId(pub u32);
#[derive(Clone, Debug)]
pub struct EntryMeta {
pub kind: u32,
pub attr1: u32,
pub attr2: u32,
pub attr3: u32,
pub name: String,
pub data_offset: u64,
pub data_size: u32,
pub sort_index: u32,
}
#[derive(Copy, Clone, Debug)]
pub struct EntryRef<'a> {
pub id: EntryId,
pub meta: &'a EntryMeta,
}
#[derive(Clone, Debug)]
struct EntryRecord {
meta: EntryMeta,
name_raw: [u8; 36],
}
impl Archive {
pub fn open_path(path: impl AsRef<Path>) -> Result<Self> {
Self::open_path_with(path, OpenMode::ReadOnly, OpenOptions::default())
}
pub fn open_path_with(
path: impl AsRef<Path>,
_mode: OpenMode,
opts: OpenOptions,
) -> Result<Self> {
let bytes = fs::read(path.as_ref())?;
let arc: Arc<[u8]> = Arc::from(bytes.into_boxed_slice());
Self::open_bytes(arc, opts)
}
pub fn open_bytes(bytes: Arc<[u8]>, opts: OpenOptions) -> Result<Self> {
let (entries, _) = parse_archive(&bytes, opts.raw_mode)?;
if opts.prefetch_pages {
prefetch_pages(&bytes);
}
Ok(Self {
bytes,
entries,
raw_mode: opts.raw_mode,
})
}
pub fn entry_count(&self) -> usize {
self.entries.len()
}
pub fn entries(&self) -> impl Iterator<Item = EntryRef<'_>> {
self.entries
.iter()
.enumerate()
.map(|(idx, entry)| EntryRef {
id: EntryId(u32::try_from(idx).expect("entry count validated at parse")),
meta: &entry.meta,
})
}
pub fn find(&self, name: &str) -> Option<EntryId> {
if self.entries.is_empty() {
return None;
}
if !self.raw_mode {
let mut low = 0usize;
let mut high = self.entries.len();
while low < high {
let mid = low + (high - low) / 2;
let Ok(target_idx) = usize::try_from(self.entries[mid].meta.sort_index) else {
break;
};
if target_idx >= self.entries.len() {
break;
}
let cmp = cmp_name_case_insensitive(
name.as_bytes(),
entry_name_bytes(&self.entries[target_idx].name_raw),
);
match cmp {
Ordering::Less => high = mid,
Ordering::Greater => low = mid + 1,
Ordering::Equal => {
return Some(EntryId(
u32::try_from(target_idx).expect("entry count validated at parse"),
))
}
}
}
}
self.entries.iter().enumerate().find_map(|(idx, entry)| {
if cmp_name_case_insensitive(name.as_bytes(), entry_name_bytes(&entry.name_raw))
== Ordering::Equal
{
Some(EntryId(
u32::try_from(idx).expect("entry count validated at parse"),
))
} else {
None
}
})
}
pub fn get(&self, id: EntryId) -> Option<EntryRef<'_>> {
let idx = usize::try_from(id.0).ok()?;
let entry = self.entries.get(idx)?;
Some(EntryRef {
id,
meta: &entry.meta,
})
}
pub fn read(&self, id: EntryId) -> Result<ResourceData<'_>> {
let range = self.entry_range(id)?;
Ok(ResourceData::Borrowed(&self.bytes[range]))
}
pub fn read_into(&self, id: EntryId, out: &mut dyn OutputBuffer) -> Result<usize> {
let range = self.entry_range(id)?;
out.write_exact(&self.bytes[range.clone()])?;
Ok(range.len())
}
pub fn raw_slice(&self, id: EntryId) -> Result<Option<&[u8]>> {
let range = self.entry_range(id)?;
Ok(Some(&self.bytes[range]))
}
pub fn edit_path(path: impl AsRef<Path>) -> Result<Editor> {
let path_buf = path.as_ref().to_path_buf();
let bytes = fs::read(&path_buf)?;
let arc: Arc<[u8]> = Arc::from(bytes.into_boxed_slice());
let (entries, _) = parse_archive(&arc, false)?;
let mut editable = Vec::with_capacity(entries.len());
for entry in &entries {
let range = checked_range(entry.meta.data_offset, entry.meta.data_size, arc.len())?;
editable.push(EditableEntry {
meta: entry.meta.clone(),
name_raw: entry.name_raw,
data: EntryData::Borrowed(range), // Copy-on-write: only store range
});
}
Ok(Editor {
path: path_buf,
source: arc,
entries: editable,
})
}
fn entry_range(&self, id: EntryId) -> Result<Range<usize>> {
let idx = usize::try_from(id.0).map_err(|_| Error::IntegerOverflow)?;
let Some(entry) = self.entries.get(idx) else {
return Err(Error::EntryIdOutOfRange {
id: id.0,
entry_count: self.entries.len().try_into().unwrap_or(u32::MAX),
});
};
checked_range(
entry.meta.data_offset,
entry.meta.data_size,
self.bytes.len(),
)
}
}
pub struct Editor {
path: PathBuf,
source: Arc<[u8]>,
entries: Vec<EditableEntry>,
}
#[derive(Clone, Debug)]
enum EntryData {
Borrowed(Range<usize>),
Modified(Vec<u8>),
}
#[derive(Clone, Debug)]
struct EditableEntry {
meta: EntryMeta,
name_raw: [u8; 36],
data: EntryData,
}
impl EditableEntry {
fn data_slice<'a>(&'a self, source: &'a Arc<[u8]>) -> &'a [u8] {
match &self.data {
EntryData::Borrowed(range) => &source[range.clone()],
EntryData::Modified(vec) => vec.as_slice(),
}
}
}
#[derive(Clone, Debug)]
pub struct NewEntry<'a> {
pub kind: u32,
pub attr1: u32,
pub attr2: u32,
pub attr3: u32,
pub name: &'a str,
pub data: &'a [u8],
}
impl Editor {
pub fn entries(&self) -> impl Iterator<Item = EntryRef<'_>> {
self.entries
.iter()
.enumerate()
.map(|(idx, entry)| EntryRef {
id: EntryId(u32::try_from(idx).expect("entry count validated at add")),
meta: &entry.meta,
})
}
pub fn add(&mut self, entry: NewEntry<'_>) -> Result<EntryId> {
let name_raw = encode_name_field(entry.name)?;
let id_u32 = u32::try_from(self.entries.len()).map_err(|_| Error::IntegerOverflow)?;
let data_size = u32::try_from(entry.data.len()).map_err(|_| Error::IntegerOverflow)?;
self.entries.push(EditableEntry {
meta: EntryMeta {
kind: entry.kind,
attr1: entry.attr1,
attr2: entry.attr2,
attr3: entry.attr3,
name: decode_name(entry_name_bytes(&name_raw)),
data_offset: 0,
data_size,
sort_index: 0,
},
name_raw,
data: EntryData::Modified(entry.data.to_vec()),
});
Ok(EntryId(id_u32))
}
pub fn replace_data(&mut self, id: EntryId, data: &[u8]) -> Result<()> {
let idx = usize::try_from(id.0).map_err(|_| Error::IntegerOverflow)?;
let Some(entry) = self.entries.get_mut(idx) else {
return Err(Error::EntryIdOutOfRange {
id: id.0,
entry_count: self.entries.len().try_into().unwrap_or(u32::MAX),
});
};
entry.meta.data_size = u32::try_from(data.len()).map_err(|_| Error::IntegerOverflow)?;
// Replace with new data (triggers copy-on-write if borrowed)
entry.data = EntryData::Modified(data.to_vec());
Ok(())
}
pub fn remove(&mut self, id: EntryId) -> Result<()> {
let idx = usize::try_from(id.0).map_err(|_| Error::IntegerOverflow)?;
if idx >= self.entries.len() {
return Err(Error::EntryIdOutOfRange {
id: id.0,
entry_count: self.entries.len().try_into().unwrap_or(u32::MAX),
});
}
self.entries.remove(idx);
Ok(())
}
pub fn commit(mut self) -> Result<()> {
let count_u32 = u32::try_from(self.entries.len()).map_err(|_| Error::IntegerOverflow)?;
// Pre-calculate capacity to avoid reallocations
let total_data_size: usize = self
.entries
.iter()
.map(|e| e.data_slice(&self.source).len())
.sum();
let padding_estimate = self.entries.len() * 8; // Max 8 bytes padding per entry
let directory_size = self.entries.len() * 64; // 64 bytes per entry
let capacity = 16 + total_data_size + padding_estimate + directory_size;
let mut out = Vec::with_capacity(capacity);
out.resize(16, 0); // Header
// Keep reference to source for copy-on-write
let source = &self.source;
for entry in &mut self.entries {
entry.meta.data_offset =
u64::try_from(out.len()).map_err(|_| Error::IntegerOverflow)?;
// Calculate size and get slice separately to avoid borrow conflicts
let data_len = entry.data_slice(source).len();
entry.meta.data_size = u32::try_from(data_len).map_err(|_| Error::IntegerOverflow)?;
// Now get the slice again for writing
let data_slice = entry.data_slice(source);
out.extend_from_slice(data_slice);
let padding = (8 - (out.len() % 8)) % 8;
if padding > 0 {
out.resize(out.len() + padding, 0);
}
}
let mut sort_order: Vec<usize> = (0..self.entries.len()).collect();
sort_order.sort_by(|a, b| {
cmp_name_case_insensitive(
entry_name_bytes(&self.entries[*a].name_raw),
entry_name_bytes(&self.entries[*b].name_raw),
)
});
for (idx, entry) in self.entries.iter_mut().enumerate() {
entry.meta.sort_index =
u32::try_from(sort_order[idx]).map_err(|_| Error::IntegerOverflow)?;
}
for entry in &self.entries {
let data_offset_u32 =
u32::try_from(entry.meta.data_offset).map_err(|_| Error::IntegerOverflow)?;
push_u32(&mut out, entry.meta.kind);
push_u32(&mut out, entry.meta.attr1);
push_u32(&mut out, entry.meta.attr2);
push_u32(&mut out, entry.meta.data_size);
push_u32(&mut out, entry.meta.attr3);
out.extend_from_slice(&entry.name_raw);
push_u32(&mut out, data_offset_u32);
push_u32(&mut out, entry.meta.sort_index);
}
let total_size_u32 = u32::try_from(out.len()).map_err(|_| Error::IntegerOverflow)?;
out[0..4].copy_from_slice(b"NRes");
out[4..8].copy_from_slice(&0x100_u32.to_le_bytes());
out[8..12].copy_from_slice(&count_u32.to_le_bytes());
out[12..16].copy_from_slice(&total_size_u32.to_le_bytes());
write_atomic(&self.path, &out)
}
}
fn parse_archive(bytes: &[u8], raw_mode: bool) -> Result<(Vec<EntryRecord>, u64)> {
if raw_mode {
let data_size = u32::try_from(bytes.len()).map_err(|_| Error::IntegerOverflow)?;
let entry = EntryRecord {
meta: EntryMeta {
kind: 0,
attr1: 0,
attr2: 0,
attr3: 0,
name: String::from("RAW"),
data_offset: 0,
data_size,
sort_index: 0,
},
name_raw: {
let mut name = [0u8; 36];
let bytes_name = b"RAW";
name[..bytes_name.len()].copy_from_slice(bytes_name);
name
},
};
return Ok((
vec![entry],
u64::try_from(bytes.len()).map_err(|_| Error::IntegerOverflow)?,
));
}
if bytes.len() < 16 {
let mut got = [0u8; 4];
let copy_len = bytes.len().min(4);
got[..copy_len].copy_from_slice(&bytes[..copy_len]);
return Err(Error::InvalidMagic { got });
}
let mut magic = [0u8; 4];
magic.copy_from_slice(&bytes[0..4]);
if &magic != b"NRes" {
return Err(Error::InvalidMagic { got: magic });
}
let version = read_u32(bytes, 4)?;
if version != 0x100 {
return Err(Error::UnsupportedVersion { got: version });
}
let entry_count_i32 = i32::from_le_bytes(
bytes[8..12]
.try_into()
.map_err(|_| Error::IntegerOverflow)?,
);
if entry_count_i32 < 0 {
return Err(Error::InvalidEntryCount {
got: entry_count_i32,
});
}
let entry_count = usize::try_from(entry_count_i32).map_err(|_| Error::IntegerOverflow)?;
// Validate entry_count fits in u32 (required for EntryId)
if entry_count > u32::MAX as usize {
return Err(Error::TooManyEntries { got: entry_count });
}
let total_size = read_u32(bytes, 12)?;
let actual_size = u64::try_from(bytes.len()).map_err(|_| Error::IntegerOverflow)?;
if u64::from(total_size) != actual_size {
return Err(Error::TotalSizeMismatch {
header: total_size,
actual: actual_size,
});
}
let directory_len = u64::try_from(entry_count)
.map_err(|_| Error::IntegerOverflow)?
.checked_mul(64)
.ok_or(Error::IntegerOverflow)?;
let directory_offset =
u64::from(total_size)
.checked_sub(directory_len)
.ok_or(Error::DirectoryOutOfBounds {
directory_offset: 0,
directory_len,
file_len: actual_size,
})?;
if directory_offset < 16 || directory_offset + directory_len > actual_size {
return Err(Error::DirectoryOutOfBounds {
directory_offset,
directory_len,
file_len: actual_size,
});
}
let mut entries = Vec::with_capacity(entry_count);
for index in 0..entry_count {
let base = usize::try_from(directory_offset)
.map_err(|_| Error::IntegerOverflow)?
.checked_add(index.checked_mul(64).ok_or(Error::IntegerOverflow)?)
.ok_or(Error::IntegerOverflow)?;
let kind = read_u32(bytes, base)?;
let attr1 = read_u32(bytes, base + 4)?;
let attr2 = read_u32(bytes, base + 8)?;
let data_size = read_u32(bytes, base + 12)?;
let attr3 = read_u32(bytes, base + 16)?;
let mut name_raw = [0u8; 36];
let name_slice = bytes
.get(base + 20..base + 56)
.ok_or(Error::IntegerOverflow)?;
name_raw.copy_from_slice(name_slice);
let name_bytes = entry_name_bytes(&name_raw);
if name_bytes.len() > 35 {
return Err(Error::NameTooLong {
got: name_bytes.len(),
max: 35,
});
}
let data_offset = u64::from(read_u32(bytes, base + 56)?);
let sort_index = read_u32(bytes, base + 60)?;
let end = data_offset
.checked_add(u64::from(data_size))
.ok_or(Error::IntegerOverflow)?;
if data_offset < 16 || end > directory_offset {
return Err(Error::EntryDataOutOfBounds {
id: u32::try_from(index).map_err(|_| Error::IntegerOverflow)?,
offset: data_offset,
size: data_size,
directory_offset,
});
}
entries.push(EntryRecord {
meta: EntryMeta {
kind,
attr1,
attr2,
attr3,
name: decode_name(name_bytes),
data_offset,
data_size,
sort_index,
},
name_raw,
});
}
Ok((entries, directory_offset))
}
fn checked_range(offset: u64, size: u32, bytes_len: usize) -> Result<Range<usize>> {
let start = usize::try_from(offset).map_err(|_| Error::IntegerOverflow)?;
let len = usize::try_from(size).map_err(|_| Error::IntegerOverflow)?;
let end = start.checked_add(len).ok_or(Error::IntegerOverflow)?;
if end > bytes_len {
return Err(Error::IntegerOverflow);
}
Ok(start..end)
}
fn read_u32(bytes: &[u8], offset: usize) -> Result<u32> {
let data = bytes
.get(offset..offset + 4)
.ok_or(Error::IntegerOverflow)?;
let arr: [u8; 4] = data.try_into().map_err(|_| Error::IntegerOverflow)?;
Ok(u32::from_le_bytes(arr))
}
fn push_u32(out: &mut Vec<u8>, value: u32) {
out.extend_from_slice(&value.to_le_bytes());
}
fn encode_name_field(name: &str) -> Result<[u8; 36]> {
let bytes = name.as_bytes();
if bytes.contains(&0) {
return Err(Error::NameContainsNul);
}
if bytes.len() > 35 {
return Err(Error::NameTooLong {
got: bytes.len(),
max: 35,
});
}
let mut out = [0u8; 36];
out[..bytes.len()].copy_from_slice(bytes);
Ok(out)
}
fn entry_name_bytes(raw: &[u8; 36]) -> &[u8] {
let len = raw.iter().position(|&b| b == 0).unwrap_or(raw.len());
&raw[..len]
}
fn decode_name(name: &[u8]) -> String {
name.iter().map(|b| char::from(*b)).collect()
}
fn cmp_name_case_insensitive(a: &[u8], b: &[u8]) -> Ordering {
let mut idx = 0usize;
let min_len = a.len().min(b.len());
while idx < min_len {
let left = ascii_lower(a[idx]);
let right = ascii_lower(b[idx]);
if left != right {
return left.cmp(&right);
}
idx += 1;
}
a.len().cmp(&b.len())
}
fn ascii_lower(value: u8) -> u8 {
if value.is_ascii_uppercase() {
value + 32
} else {
value
}
}
fn prefetch_pages(bytes: &[u8]) {
use std::sync::atomic::{compiler_fence, Ordering};
let mut cursor = 0usize;
let mut sink = 0u8;
while cursor < bytes.len() {
sink ^= bytes[cursor];
cursor = cursor.saturating_add(4096);
}
compiler_fence(Ordering::SeqCst);
let _ = sink;
}
fn write_atomic(path: &Path, content: &[u8]) -> Result<()> {
let file_name = path
.file_name()
.and_then(|name| name.to_str())
.unwrap_or("archive");
let parent = path.parent().unwrap_or_else(|| Path::new("."));
let mut temp_path = None;
for attempt in 0..128u32 {
let name = format!(
".{}.tmp.{}.{}.{}",
file_name,
std::process::id(),
unix_time_nanos(),
attempt
);
let candidate = parent.join(name);
let opened = FsOpenOptions::new()
.create_new(true)
.write(true)
.open(&candidate);
if let Ok(mut file) = opened {
file.write_all(content)?;
file.sync_all()?;
temp_path = Some((candidate, file));
break;
}
}
let Some((tmp_path, mut file)) = temp_path else {
return Err(Error::Io(std::io::Error::new(
std::io::ErrorKind::AlreadyExists,
"failed to create temporary file for atomic write",
)));
};
file.flush()?;
drop(file);
if let Err(err) = replace_file_atomically(&tmp_path, path) {
let _ = fs::remove_file(&tmp_path);
return Err(Error::Io(err));
}
Ok(())
}
#[cfg(not(windows))]
fn replace_file_atomically(src: &Path, dst: &Path) -> std::io::Result<()> {
fs::rename(src, dst)
}
#[cfg(windows)]
fn replace_file_atomically(src: &Path, dst: &Path) -> std::io::Result<()> {
use std::iter;
use std::os::windows::ffi::OsStrExt;
use windows_sys::Win32::Storage::FileSystem::{
MoveFileExW, MOVEFILE_REPLACE_EXISTING, MOVEFILE_WRITE_THROUGH,
};
let src_wide: Vec<u16> = src.as_os_str().encode_wide().chain(iter::once(0)).collect();
let dst_wide: Vec<u16> = dst.as_os_str().encode_wide().chain(iter::once(0)).collect();
// Replace destination in one OS call, avoiding remove+rename gaps on Windows.
let ok = unsafe {
MoveFileExW(
src_wide.as_ptr(),
dst_wide.as_ptr(),
MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH,
)
};
if ok == 0 {
Err(std::io::Error::last_os_error())
} else {
Ok(())
}
}
fn unix_time_nanos() -> u128 {
match SystemTime::now().duration_since(UNIX_EPOCH) {
Ok(duration) => duration.as_nanos(),
Err(_) => 0,
}
}
#[cfg(test)]
mod tests;
-996
View File
@@ -1,996 +0,0 @@
use super::*;
use std::any::Any;
use std::fs;
use std::panic::{catch_unwind, AssertUnwindSafe};
#[derive(Clone)]
struct SyntheticEntry<'a> {
kind: u32,
attr1: u32,
attr2: u32,
attr3: u32,
name: &'a str,
data: &'a [u8],
}
fn collect_files_recursive(root: &Path, out: &mut Vec<PathBuf>) {
let Ok(entries) = fs::read_dir(root) else {
return;
};
for entry in entries.flatten() {
let path = entry.path();
if path.is_dir() {
collect_files_recursive(&path, out);
} else if path.is_file() {
out.push(path);
}
}
}
fn nres_test_files() -> Vec<PathBuf> {
let root = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join("testdata")
.join("nres");
let mut files = Vec::new();
collect_files_recursive(&root, &mut files);
files.sort();
files
.into_iter()
.filter(|path| {
fs::read(path)
.map(|data| data.get(0..4) == Some(b"NRes"))
.unwrap_or(false)
})
.collect()
}
fn make_temp_copy(original: &Path, bytes: &[u8]) -> PathBuf {
let mut path = std::env::temp_dir();
let file_name = original
.file_name()
.and_then(|v| v.to_str())
.unwrap_or("archive");
path.push(format!(
"nres-test-{}-{}-{}",
std::process::id(),
unix_time_nanos(),
file_name
));
fs::write(&path, bytes).expect("failed to create temp file");
path
}
fn panic_message(payload: Box<dyn Any + Send>) -> String {
let any = payload.as_ref();
if let Some(message) = any.downcast_ref::<String>() {
return message.clone();
}
if let Some(message) = any.downcast_ref::<&str>() {
return (*message).to_string();
}
String::from("panic without message")
}
fn read_u32_le(bytes: &[u8], offset: usize) -> u32 {
let slice = bytes
.get(offset..offset + 4)
.expect("u32 read out of bounds in test");
let arr: [u8; 4] = slice.try_into().expect("u32 conversion failed in test");
u32::from_le_bytes(arr)
}
fn read_i32_le(bytes: &[u8], offset: usize) -> i32 {
let slice = bytes
.get(offset..offset + 4)
.expect("i32 read out of bounds in test");
let arr: [u8; 4] = slice.try_into().expect("i32 conversion failed in test");
i32::from_le_bytes(arr)
}
fn name_field_bytes(raw: &[u8; 36]) -> Option<&[u8]> {
let nul = raw.iter().position(|value| *value == 0)?;
Some(&raw[..nul])
}
fn build_nres_bytes(entries: &[SyntheticEntry<'_>]) -> Vec<u8> {
let mut out = vec![0u8; 16];
let mut offsets = Vec::with_capacity(entries.len());
for entry in entries {
offsets.push(u32::try_from(out.len()).expect("offset overflow"));
out.extend_from_slice(entry.data);
let padding = (8 - (out.len() % 8)) % 8;
if padding > 0 {
out.resize(out.len() + padding, 0);
}
}
let mut sort_order: Vec<usize> = (0..entries.len()).collect();
sort_order.sort_by(|a, b| {
cmp_name_case_insensitive(entries[*a].name.as_bytes(), entries[*b].name.as_bytes())
});
for (index, entry) in entries.iter().enumerate() {
let mut name_raw = [0u8; 36];
let name_bytes = entry.name.as_bytes();
assert!(name_bytes.len() <= 35, "name too long in fixture");
name_raw[..name_bytes.len()].copy_from_slice(name_bytes);
push_u32(&mut out, entry.kind);
push_u32(&mut out, entry.attr1);
push_u32(&mut out, entry.attr2);
push_u32(
&mut out,
u32::try_from(entry.data.len()).expect("data size overflow"),
);
push_u32(&mut out, entry.attr3);
out.extend_from_slice(&name_raw);
push_u32(&mut out, offsets[index]);
push_u32(
&mut out,
u32::try_from(sort_order[index]).expect("sort index overflow"),
);
}
out[0..4].copy_from_slice(b"NRes");
out[4..8].copy_from_slice(&0x100_u32.to_le_bytes());
out[8..12].copy_from_slice(
&u32::try_from(entries.len())
.expect("count overflow")
.to_le_bytes(),
);
let total_size = u32::try_from(out.len()).expect("size overflow");
out[12..16].copy_from_slice(&total_size.to_le_bytes());
out
}
#[test]
fn nres_docs_structural_invariants_all_files() {
let files = nres_test_files();
if files.is_empty() {
eprintln!(
"skipping nres_docs_structural_invariants_all_files: no NRes archives in testdata/nres"
);
return;
}
for path in files {
let bytes = fs::read(&path).unwrap_or_else(|err| {
panic!("failed to read {}: {err}", path.display());
});
assert!(
bytes.len() >= 16,
"NRes header too short in {}",
path.display()
);
assert_eq!(&bytes[0..4], b"NRes", "bad magic in {}", path.display());
assert_eq!(
read_u32_le(&bytes, 4),
0x100,
"bad version in {}",
path.display()
);
assert_eq!(
usize::try_from(read_u32_le(&bytes, 12)).expect("size overflow"),
bytes.len(),
"header.total_size mismatch in {}",
path.display()
);
let entry_count_i32 = read_i32_le(&bytes, 8);
assert!(
entry_count_i32 >= 0,
"negative entry_count={} in {}",
entry_count_i32,
path.display()
);
let entry_count = usize::try_from(entry_count_i32).expect("entry_count overflow");
let directory_len = entry_count.checked_mul(64).expect("directory_len overflow");
let directory_offset = bytes
.len()
.checked_sub(directory_len)
.unwrap_or_else(|| panic!("directory underflow in {}", path.display()));
assert!(
directory_offset >= 16,
"directory offset before data area in {}",
path.display()
);
assert_eq!(
directory_offset + directory_len,
bytes.len(),
"directory not at file end in {}",
path.display()
);
let mut sort_indices = Vec::with_capacity(entry_count);
let mut entries = Vec::with_capacity(entry_count);
for index in 0..entry_count {
let base = directory_offset + index * 64;
let size = usize::try_from(read_u32_le(&bytes, base + 12)).expect("size overflow");
let data_offset =
usize::try_from(read_u32_le(&bytes, base + 56)).expect("offset overflow");
let sort_index =
usize::try_from(read_u32_le(&bytes, base + 60)).expect("sort_index overflow");
let mut name_raw = [0u8; 36];
name_raw.copy_from_slice(
bytes
.get(base + 20..base + 56)
.expect("name field out of bounds in test"),
);
let name_bytes = name_field_bytes(&name_raw).unwrap_or_else(|| {
panic!(
"name field without NUL terminator in {} entry #{index}",
path.display()
)
});
assert!(
name_bytes.len() <= 35,
"name longer than 35 bytes in {} entry #{index}",
path.display()
);
sort_indices.push(sort_index);
entries.push((name_bytes.to_vec(), data_offset, size));
}
let mut expected_sort: Vec<usize> = (0..entry_count).collect();
expected_sort.sort_by(|a, b| cmp_name_case_insensitive(&entries[*a].0, &entries[*b].0));
assert_eq!(
sort_indices,
expected_sort,
"sort_index table mismatch in {}",
path.display()
);
let mut data_regions: Vec<(usize, usize)> =
entries.iter().map(|(_, off, size)| (*off, *size)).collect();
data_regions.sort_by_key(|(off, _)| *off);
for (idx, (data_offset, size)) in data_regions.iter().enumerate() {
assert_eq!(
data_offset % 8,
0,
"data offset is not 8-byte aligned in {} (region #{idx})",
path.display()
);
assert!(
*data_offset >= 16,
"data offset before header end in {} (region #{idx})",
path.display()
);
assert!(
data_offset.checked_add(*size).unwrap_or(usize::MAX) <= directory_offset,
"data region overlaps directory in {} (region #{idx})",
path.display()
);
}
for pair in data_regions.windows(2) {
let (start, size) = pair[0];
let (next_start, _) = pair[1];
let end = start
.checked_add(size)
.unwrap_or_else(|| panic!("size overflow in {}", path.display()));
assert!(
end <= next_start,
"overlapping data regions in {}: [{start}, {end}) and next at {next_start}",
path.display()
);
for (offset, value) in bytes[end..next_start].iter().enumerate() {
assert_eq!(
*value,
0,
"non-zero alignment padding in {} at offset {}",
path.display(),
end + offset
);
}
}
}
}
#[test]
fn nres_read_and_roundtrip_all_files() {
let files = nres_test_files();
if files.is_empty() {
eprintln!("skipping nres_read_and_roundtrip_all_files: no NRes archives in testdata/nres");
return;
}
let checked = files.len();
let mut success = 0usize;
let mut failures = Vec::new();
for path in files {
let display_path = path.display().to_string();
let result = catch_unwind(AssertUnwindSafe(|| {
let original = fs::read(&path).expect("failed to read archive");
let archive = Archive::open_path(&path)
.unwrap_or_else(|err| panic!("failed to open {}: {err}", path.display()));
let count = archive.entry_count();
assert_eq!(
count,
archive.entries().count(),
"entry count mismatch: {}",
path.display()
);
for idx in 0..count {
let id = EntryId(idx as u32);
let entry = archive
.get(id)
.unwrap_or_else(|| panic!("missing entry #{idx} in {}", path.display()));
let payload = archive.read(id).unwrap_or_else(|err| {
panic!("read failed for {} entry #{idx}: {err}", path.display())
});
let mut out = Vec::new();
let written = archive.read_into(id, &mut out).unwrap_or_else(|err| {
panic!(
"read_into failed for {} entry #{idx}: {err}",
path.display()
)
});
assert_eq!(
written,
payload.as_slice().len(),
"size mismatch in {} entry #{idx}",
path.display()
);
assert_eq!(
out.as_slice(),
payload.as_slice(),
"payload mismatch in {} entry #{idx}",
path.display()
);
let raw = archive
.raw_slice(id)
.unwrap_or_else(|err| {
panic!(
"raw_slice failed for {} entry #{idx}: {err}",
path.display()
)
})
.expect("raw_slice must return Some for file-backed archive");
assert_eq!(
raw,
payload.as_slice(),
"raw slice mismatch in {} entry #{idx}",
path.display()
);
let found = archive.find(&entry.meta.name).unwrap_or_else(|| {
panic!(
"find failed for name '{}' in {}",
entry.meta.name,
path.display()
)
});
let found_meta = archive.get(found).expect("find returned invalid id");
assert!(
found_meta.meta.name.eq_ignore_ascii_case(&entry.meta.name),
"find returned unrelated entry in {}",
path.display()
);
}
let temp_copy = make_temp_copy(&path, &original);
let mut editor = Archive::edit_path(&temp_copy)
.unwrap_or_else(|err| panic!("edit_path failed for {}: {err}", path.display()));
for idx in 0..count {
let data = archive
.read(EntryId(idx as u32))
.unwrap_or_else(|err| {
panic!(
"read before replace failed for {} entry #{idx}: {err}",
path.display()
)
})
.into_owned();
editor
.replace_data(EntryId(idx as u32), &data)
.unwrap_or_else(|err| {
panic!(
"replace_data failed for {} entry #{idx}: {err}",
path.display()
)
});
}
editor
.commit()
.unwrap_or_else(|err| panic!("commit failed for {}: {err}", path.display()));
let rebuilt = fs::read(&temp_copy).expect("failed to read rebuilt archive");
let _ = fs::remove_file(&temp_copy);
assert_eq!(
original,
rebuilt,
"byte-to-byte roundtrip mismatch for {}",
path.display()
);
}));
match result {
Ok(()) => success += 1,
Err(payload) => {
failures.push(format!("{}: {}", display_path, panic_message(payload)));
}
}
}
let failed = failures.len();
eprintln!(
"NRes summary: checked={}, success={}, failed={}",
checked, success, failed
);
if !failures.is_empty() {
panic!(
"NRes validation failed.\nsummary: checked={}, success={}, failed={}\n{}",
checked,
success,
failed,
failures.join("\n")
);
}
}
#[test]
fn nres_raw_mode_exposes_whole_file() {
let files = nres_test_files();
let Some(first) = files.first() else {
eprintln!("skipping nres_raw_mode_exposes_whole_file: no NRes archives in testdata/nres");
return;
};
let original = fs::read(first).expect("failed to read archive");
let arc: Arc<[u8]> = Arc::from(original.clone().into_boxed_slice());
let archive = Archive::open_bytes(
arc,
OpenOptions {
raw_mode: true,
sequential_hint: false,
prefetch_pages: false,
},
)
.expect("raw mode open failed");
assert_eq!(archive.entry_count(), 1);
let data = archive.read(EntryId(0)).expect("raw read failed");
assert_eq!(data.as_slice(), original.as_slice());
}
#[test]
fn nres_raw_mode_accepts_non_nres_bytes() {
let payload = b"not-an-nres-archive".to_vec();
let bytes: Arc<[u8]> = Arc::from(payload.clone().into_boxed_slice());
match Archive::open_bytes(bytes.clone(), OpenOptions::default()) {
Err(Error::InvalidMagic { .. }) => {}
other => panic!("expected InvalidMagic without raw_mode, got {other:?}"),
}
let archive = Archive::open_bytes(
bytes,
OpenOptions {
raw_mode: true,
sequential_hint: false,
prefetch_pages: false,
},
)
.expect("raw_mode should accept any bytes");
assert_eq!(archive.entry_count(), 1);
assert_eq!(archive.find("raw"), Some(EntryId(0)));
assert_eq!(
archive
.read(EntryId(0))
.expect("raw read failed")
.as_slice(),
payload.as_slice()
);
}
#[test]
fn nres_open_options_hints_do_not_change_payload() {
let payload: Vec<u8> = (0..70_000u32).map(|v| (v % 251) as u8).collect();
let src = build_nres_bytes(&[SyntheticEntry {
kind: 7,
attr1: 70,
attr2: 700,
attr3: 7000,
name: "big.bin",
data: &payload,
}]);
let arc: Arc<[u8]> = Arc::from(src.into_boxed_slice());
let baseline = Archive::open_bytes(arc.clone(), OpenOptions::default())
.expect("baseline open should succeed");
let hinted = Archive::open_bytes(
arc,
OpenOptions {
raw_mode: false,
sequential_hint: true,
prefetch_pages: true,
},
)
.expect("open with hints should succeed");
assert_eq!(baseline.entry_count(), 1);
assert_eq!(hinted.entry_count(), 1);
assert_eq!(baseline.find("BIG.BIN"), Some(EntryId(0)));
assert_eq!(hinted.find("big.bin"), Some(EntryId(0)));
assert_eq!(
baseline
.read(EntryId(0))
.expect("baseline read failed")
.as_slice(),
hinted
.read(EntryId(0))
.expect("hinted read failed")
.as_slice()
);
}
#[test]
fn nres_commit_empty_archive_has_minimal_layout() {
let mut path = std::env::temp_dir();
path.push(format!(
"nres-empty-commit-{}-{}.lib",
std::process::id(),
unix_time_nanos()
));
fs::write(&path, build_nres_bytes(&[])).expect("write empty archive failed");
Archive::edit_path(&path)
.expect("edit_path failed for empty archive")
.commit()
.expect("commit failed for empty archive");
let bytes = fs::read(&path).expect("failed to read committed archive");
assert_eq!(bytes.len(), 16, "empty archive must contain only header");
assert_eq!(&bytes[0..4], b"NRes");
assert_eq!(read_u32_le(&bytes, 4), 0x100);
assert_eq!(read_u32_le(&bytes, 8), 0);
assert_eq!(read_u32_le(&bytes, 12), 16);
let _ = fs::remove_file(&path);
}
#[test]
fn nres_commit_recomputes_header_directory_and_sort_table() {
let mut path = std::env::temp_dir();
path.push(format!(
"nres-commit-layout-{}-{}.lib",
std::process::id(),
unix_time_nanos()
));
fs::write(&path, build_nres_bytes(&[])).expect("write empty archive failed");
let mut editor = Archive::edit_path(&path).expect("edit_path failed");
editor
.add(NewEntry {
kind: 10,
attr1: 1,
attr2: 2,
attr3: 3,
name: "Zulu",
data: b"aaaaa",
})
.expect("add #0 failed");
editor
.add(NewEntry {
kind: 11,
attr1: 4,
attr2: 5,
attr3: 6,
name: "alpha",
data: b"bbbbbbbb",
})
.expect("add #1 failed");
editor
.add(NewEntry {
kind: 12,
attr1: 7,
attr2: 8,
attr3: 9,
name: "Beta",
data: b"cccc",
})
.expect("add #2 failed");
editor.commit().expect("commit failed");
let bytes = fs::read(&path).expect("failed to read committed archive");
assert_eq!(&bytes[0..4], b"NRes");
assert_eq!(read_u32_le(&bytes, 4), 0x100);
let entry_count = usize::try_from(read_u32_le(&bytes, 8)).expect("entry_count overflow");
let total_size = usize::try_from(read_u32_le(&bytes, 12)).expect("total_size overflow");
assert_eq!(entry_count, 3);
assert_eq!(total_size, bytes.len());
let directory_offset = total_size
.checked_sub(entry_count * 64)
.expect("invalid directory offset");
assert!(directory_offset >= 16);
let mut sort_indices = Vec::new();
let mut prev_data_end = 16usize;
for idx in 0..entry_count {
let base = directory_offset + idx * 64;
let data_size = usize::try_from(read_u32_le(&bytes, base + 12)).expect("size overflow");
let data_offset = usize::try_from(read_u32_le(&bytes, base + 56)).expect("offset overflow");
let sort_index =
usize::try_from(read_u32_le(&bytes, base + 60)).expect("sort index overflow");
assert_eq!(
data_offset % 8,
0,
"entry #{idx} data offset must be 8-byte aligned"
);
assert!(
data_offset >= prev_data_end,
"entry #{idx} offset regressed"
);
assert!(
data_offset + data_size <= directory_offset,
"entry #{idx} overlaps directory"
);
prev_data_end = data_offset + data_size;
sort_indices.push(sort_index);
}
let names = ["Zulu", "alpha", "Beta"];
let mut expected_sort: Vec<usize> = (0..names.len()).collect();
expected_sort
.sort_by(|a, b| cmp_name_case_insensitive(names[*a].as_bytes(), names[*b].as_bytes()));
assert_eq!(
sort_indices, expected_sort,
"sort table must contain original indexes in case-insensitive alphabetical order"
);
let archive = Archive::open_path(&path).expect("re-open failed");
assert_eq!(archive.find("zulu"), Some(EntryId(0)));
assert_eq!(archive.find("ALPHA"), Some(EntryId(1)));
assert_eq!(archive.find("beta"), Some(EntryId(2)));
let _ = fs::remove_file(&path);
}
#[test]
fn nres_synthetic_read_find_and_edit() {
let payload_a = b"alpha";
let payload_b = b"B";
let payload_c = b"";
let src = build_nres_bytes(&[
SyntheticEntry {
kind: 1,
attr1: 10,
attr2: 20,
attr3: 30,
name: "Alpha.TXT",
data: payload_a,
},
SyntheticEntry {
kind: 2,
attr1: 11,
attr2: 21,
attr3: 31,
name: "beta.bin",
data: payload_b,
},
SyntheticEntry {
kind: 3,
attr1: 12,
attr2: 22,
attr3: 32,
name: "Gamma",
data: payload_c,
},
]);
let archive = Archive::open_bytes(
Arc::from(src.clone().into_boxed_slice()),
OpenOptions::default(),
)
.expect("open synthetic nres failed");
assert_eq!(archive.entry_count(), 3);
assert_eq!(archive.find("alpha.txt"), Some(EntryId(0)));
assert_eq!(archive.find("BETA.BIN"), Some(EntryId(1)));
assert_eq!(archive.find("gAmMa"), Some(EntryId(2)));
assert_eq!(archive.find("missing"), None);
assert_eq!(
archive.read(EntryId(0)).expect("read #0 failed").as_slice(),
payload_a
);
assert_eq!(
archive.read(EntryId(1)).expect("read #1 failed").as_slice(),
payload_b
);
assert_eq!(
archive.read(EntryId(2)).expect("read #2 failed").as_slice(),
payload_c
);
let mut path = std::env::temp_dir();
path.push(format!(
"nres-synth-edit-{}-{}.lib",
std::process::id(),
unix_time_nanos()
));
fs::write(&path, &src).expect("write temp synthetic archive failed");
let mut editor = Archive::edit_path(&path).expect("edit_path on synthetic archive failed");
editor
.replace_data(EntryId(1), b"replaced")
.expect("replace_data failed");
let added = editor
.add(NewEntry {
kind: 4,
attr1: 13,
attr2: 23,
attr3: 33,
name: "delta",
data: b"new payload",
})
.expect("add failed");
assert_eq!(added, EntryId(3));
editor.remove(EntryId(2)).expect("remove failed");
editor.commit().expect("commit failed");
let edited = Archive::open_path(&path).expect("re-open edited archive failed");
assert_eq!(edited.entry_count(), 3);
assert_eq!(
edited
.read(edited.find("beta.bin").expect("find beta.bin failed"))
.expect("read beta.bin failed")
.as_slice(),
b"replaced"
);
assert_eq!(
edited
.read(edited.find("delta").expect("find delta failed"))
.expect("read delta failed")
.as_slice(),
b"new payload"
);
assert_eq!(edited.find("gamma"), None);
let _ = fs::remove_file(&path);
}
#[test]
fn nres_max_name_length_roundtrip() {
let max_name = "12345678901234567890123456789012345";
assert_eq!(max_name.len(), 35);
let src = build_nres_bytes(&[SyntheticEntry {
kind: 9,
attr1: 1,
attr2: 2,
attr3: 3,
name: max_name,
data: b"payload",
}]);
let archive = Archive::open_bytes(Arc::from(src.into_boxed_slice()), OpenOptions::default())
.expect("open synthetic nres failed");
assert_eq!(archive.entry_count(), 1);
assert_eq!(archive.find(max_name), Some(EntryId(0)));
assert_eq!(
archive.find(&max_name.to_ascii_lowercase()),
Some(EntryId(0))
);
let entry = archive.get(EntryId(0)).expect("missing entry 0");
assert_eq!(entry.meta.name, max_name);
assert_eq!(
archive
.read(EntryId(0))
.expect("read payload failed")
.as_slice(),
b"payload"
);
}
#[test]
fn nres_find_falls_back_when_sort_index_is_out_of_range() {
let mut bytes = build_nres_bytes(&[
SyntheticEntry {
kind: 1,
attr1: 0,
attr2: 0,
attr3: 0,
name: "Alpha",
data: b"a",
},
SyntheticEntry {
kind: 2,
attr1: 0,
attr2: 0,
attr3: 0,
name: "Beta",
data: b"b",
},
SyntheticEntry {
kind: 3,
attr1: 0,
attr2: 0,
attr3: 0,
name: "Gamma",
data: b"c",
},
]);
let entry_count = 3usize;
let directory_offset = bytes
.len()
.checked_sub(entry_count * 64)
.expect("directory offset underflow");
let mid_entry_sort_index = directory_offset + 64 + 60;
bytes[mid_entry_sort_index..mid_entry_sort_index + 4].copy_from_slice(&u32::MAX.to_le_bytes());
let archive = Archive::open_bytes(Arc::from(bytes.into_boxed_slice()), OpenOptions::default())
.expect("open archive with corrupted sort index failed");
assert_eq!(archive.find("alpha"), Some(EntryId(0)));
assert_eq!(archive.find("BETA"), Some(EntryId(1)));
assert_eq!(archive.find("gamma"), Some(EntryId(2)));
assert_eq!(archive.find("missing"), None);
}
#[test]
fn nres_validation_error_cases() {
let valid = build_nres_bytes(&[SyntheticEntry {
kind: 1,
attr1: 2,
attr2: 3,
attr3: 4,
name: "ok",
data: b"1234",
}]);
let mut invalid_magic = valid.clone();
invalid_magic[0..4].copy_from_slice(b"FAIL");
match Archive::open_bytes(
Arc::from(invalid_magic.into_boxed_slice()),
OpenOptions::default(),
) {
Err(Error::InvalidMagic { .. }) => {}
other => panic!("expected InvalidMagic, got {other:?}"),
}
let mut invalid_version = valid.clone();
invalid_version[4..8].copy_from_slice(&0x200_u32.to_le_bytes());
match Archive::open_bytes(
Arc::from(invalid_version.into_boxed_slice()),
OpenOptions::default(),
) {
Err(Error::UnsupportedVersion { got }) => assert_eq!(got, 0x200),
other => panic!("expected UnsupportedVersion, got {other:?}"),
}
let mut bad_total = valid.clone();
bad_total[12..16].copy_from_slice(&0_u32.to_le_bytes());
match Archive::open_bytes(
Arc::from(bad_total.into_boxed_slice()),
OpenOptions::default(),
) {
Err(Error::TotalSizeMismatch { .. }) => {}
other => panic!("expected TotalSizeMismatch, got {other:?}"),
}
let mut bad_count = valid.clone();
bad_count[8..12].copy_from_slice(&(-1_i32).to_le_bytes());
match Archive::open_bytes(
Arc::from(bad_count.into_boxed_slice()),
OpenOptions::default(),
) {
Err(Error::InvalidEntryCount { got }) => assert_eq!(got, -1),
other => panic!("expected InvalidEntryCount, got {other:?}"),
}
let mut bad_dir = valid.clone();
bad_dir[8..12].copy_from_slice(&1000_u32.to_le_bytes());
match Archive::open_bytes(
Arc::from(bad_dir.into_boxed_slice()),
OpenOptions::default(),
) {
Err(Error::DirectoryOutOfBounds { .. }) => {}
other => panic!("expected DirectoryOutOfBounds, got {other:?}"),
}
let mut long_name = valid.clone();
let entry_base = long_name.len() - 64;
for b in &mut long_name[entry_base + 20..entry_base + 56] {
*b = b'X';
}
match Archive::open_bytes(
Arc::from(long_name.into_boxed_slice()),
OpenOptions::default(),
) {
Err(Error::NameTooLong { .. }) => {}
other => panic!("expected NameTooLong, got {other:?}"),
}
let mut bad_data = valid.clone();
bad_data[entry_base + 56..entry_base + 60].copy_from_slice(&12_u32.to_le_bytes());
bad_data[entry_base + 12..entry_base + 16].copy_from_slice(&32_u32.to_le_bytes());
match Archive::open_bytes(
Arc::from(bad_data.into_boxed_slice()),
OpenOptions::default(),
) {
Err(Error::EntryDataOutOfBounds { .. }) => {}
other => panic!("expected EntryDataOutOfBounds, got {other:?}"),
}
let archive = Archive::open_bytes(Arc::from(valid.into_boxed_slice()), OpenOptions::default())
.expect("open valid archive failed");
match archive.read(EntryId(99)) {
Err(Error::EntryIdOutOfRange { .. }) => {}
other => panic!("expected EntryIdOutOfRange, got {other:?}"),
}
}
#[test]
fn nres_editor_validation_error_cases() {
let mut path = std::env::temp_dir();
path.push(format!(
"nres-editor-errors-{}-{}.lib",
std::process::id(),
unix_time_nanos()
));
let src = build_nres_bytes(&[]);
fs::write(&path, src).expect("write empty archive failed");
let mut editor = Archive::edit_path(&path).expect("edit_path failed");
let long_name = "X".repeat(36);
match editor.add(NewEntry {
kind: 0,
attr1: 0,
attr2: 0,
attr3: 0,
name: &long_name,
data: b"",
}) {
Err(Error::NameTooLong { .. }) => {}
other => panic!("expected NameTooLong, got {other:?}"),
}
match editor.add(NewEntry {
kind: 0,
attr1: 0,
attr2: 0,
attr3: 0,
name: "bad\0name",
data: b"",
}) {
Err(Error::NameContainsNul) => {}
other => panic!("expected NameContainsNul, got {other:?}"),
}
match editor.replace_data(EntryId(0), b"x") {
Err(Error::EntryIdOutOfRange { .. }) => {}
other => panic!("expected EntryIdOutOfRange, got {other:?}"),
}
match editor.remove(EntryId(0)) {
Err(Error::EntryIdOutOfRange { .. }) => {}
other => panic!("expected EntryIdOutOfRange, got {other:?}"),
}
let _ = fs::remove_file(&path);
}
-8
View File
@@ -1,8 +0,0 @@
[package]
name = "rsli"
version = "0.1.0"
edition = "2021"
[dependencies]
common = { path = "../common" }
flate2 = { version = "1", default-features = false, features = ["rust_backend"] }
-58
View File
@@ -1,58 +0,0 @@
# rsli
Rust-библиотека для чтения архивов формата **RsLi**.
## Что умеет
- Открытие библиотеки из файла (`open_path`, `open_path_with`).
- Дешифрование таблицы записей (XOR stream cipher).
- Поддержка AO-трейлера и media overlay (`allow_ao_trailer`).
- Поддержка quirk для Deflate `EOF+1` (`allow_deflate_eof_plus_one`).
- Поиск по имени (`find`, c приведением запроса к uppercase).
- Загрузка данных:
- `load`, `load_into`, `load_packed`, `unpack`, `load_fast`.
## Поддерживаемые методы упаковки
- `0x000` None
- `0x020` XorOnly
- `0x040` Lzss
- `0x060` XorLzss
- `0x080` LzssHuffman
- `0x0A0` XorLzssHuffman
- `0x100` Deflate
## Модель ошибок
Типизированные ошибки без паник в production-коде (`InvalidMagic`, `UnsupportedVersion`, `EntryTableOutOfBounds`, `PackedSizePastEof`, `DeflateEofPlusOneQuirkRejected`, `UnsupportedMethod`, и др.).
## Покрытие тестами
### Реальные файлы
- Рекурсивный прогон по `testdata/rsli/**`.
- Сейчас в наборе: **2 архива**.
- На реальных данных подтверждены и проходят byte-to-byte проверки методы:
- `0x040` (LZSS)
- `0x100` (Deflate)
- Для каждого архива проверяется:
- `load`/`load_into`/`load_packed`/`unpack`/`load_fast`;
- `find`;
- пересборка и сравнение **byte-to-byte**.
### Синтетические тесты
Из-за отсутствия реальных файлов для части методов добавлены синтетические архивы и тесты:
- Методы:
- `0x000`, `0x020`, `0x060`, `0x080`, `0x0A0`.
- Спецкейсы формата:
- AO trailer + overlay;
- Deflate `EOF+1` (оба режима: accepted/rejected);
- некорректные заголовки/таблицы/смещения/методы.
## Быстрый запуск тестов
```bash
cargo test -p rsli -- --nocapture
```
-14
View File
@@ -1,14 +0,0 @@
use crate::error::Error;
use crate::Result;
use flate2::read::DeflateDecoder;
use std::io::Read;
/// Decode raw Deflate (RFC 1951) payload.
pub fn decode_deflate(packed: &[u8]) -> Result<Vec<u8>> {
let mut out = Vec::new();
let mut decoder = DeflateDecoder::new(packed);
decoder
.read_to_end(&mut out)
.map_err(|_| Error::DecompressionFailed("deflate"))?;
Ok(out)
}
-298
View File
@@ -1,298 +0,0 @@
use super::xor::XorState;
use crate::error::Error;
use crate::Result;
pub(crate) const LZH_N: usize = 4096;
pub(crate) const LZH_F: usize = 60;
pub(crate) const LZH_THRESHOLD: usize = 2;
pub(crate) const LZH_N_CHAR: usize = 256 - LZH_THRESHOLD + LZH_F;
pub(crate) const LZH_T: usize = LZH_N_CHAR * 2 - 1;
pub(crate) const LZH_R: usize = LZH_T - 1;
pub(crate) const LZH_MAX_FREQ: u16 = 0x8000;
/// LZSS-Huffman decompression with optional on-the-fly XOR decryption.
pub fn lzss_huffman_decompress(
data: &[u8],
expected_size: usize,
xor_key: Option<u16>,
) -> Result<Vec<u8>> {
let mut decoder = LzhDecoder::new(data, xor_key);
decoder.decode(expected_size)
}
struct LzhDecoder<'a> {
bit_reader: BitReader<'a>,
text: [u8; LZH_N],
freq: [u16; LZH_T + 1],
parent: [usize; LZH_T + LZH_N_CHAR],
son: [usize; LZH_T],
d_code: [u8; 256],
d_len: [u8; 256],
ring_pos: usize,
}
impl<'a> LzhDecoder<'a> {
fn new(data: &'a [u8], xor_key: Option<u16>) -> Self {
let mut decoder = Self {
bit_reader: BitReader::new(data, xor_key),
text: [0x20u8; LZH_N],
freq: [0u16; LZH_T + 1],
parent: [0usize; LZH_T + LZH_N_CHAR],
son: [0usize; LZH_T],
d_code: [0u8; 256],
d_len: [0u8; 256],
ring_pos: LZH_N - LZH_F,
};
decoder.init_tables();
decoder.start_huff();
decoder
}
fn decode(&mut self, expected_size: usize) -> Result<Vec<u8>> {
let mut out = Vec::with_capacity(expected_size);
while out.len() < expected_size {
let c = self.decode_char()?;
if c < 256 {
let byte = c as u8;
out.push(byte);
self.text[self.ring_pos] = byte;
self.ring_pos = (self.ring_pos + 1) & (LZH_N - 1);
} else {
let mut offset = self.decode_position()?;
offset = (self.ring_pos.wrapping_sub(offset).wrapping_sub(1)) & (LZH_N - 1);
let mut length = c.saturating_sub(253);
while length > 0 && out.len() < expected_size {
let byte = self.text[offset];
out.push(byte);
self.text[self.ring_pos] = byte;
self.ring_pos = (self.ring_pos + 1) & (LZH_N - 1);
offset = (offset + 1) & (LZH_N - 1);
length -= 1;
}
}
}
if out.len() != expected_size {
return Err(Error::DecompressionFailed("lzss-huffman"));
}
Ok(out)
}
fn init_tables(&mut self) {
let d_code_group_counts = [1usize, 3, 8, 12, 24, 16];
let d_len_group_counts = [32usize, 48, 64, 48, 48, 16];
let mut group_index = 0u8;
let mut idx = 0usize;
let mut run = 32usize;
for count in d_code_group_counts {
for _ in 0..count {
for _ in 0..run {
self.d_code[idx] = group_index;
idx += 1;
}
group_index = group_index.wrapping_add(1);
}
run >>= 1;
}
let mut len = 3u8;
idx = 0;
for count in d_len_group_counts {
for _ in 0..count {
self.d_len[idx] = len;
idx += 1;
}
len = len.saturating_add(1);
}
}
fn start_huff(&mut self) {
for i in 0..LZH_N_CHAR {
self.freq[i] = 1;
self.son[i] = i + LZH_T;
self.parent[i + LZH_T] = i;
}
let mut i = 0usize;
let mut j = LZH_N_CHAR;
while j <= LZH_R {
self.freq[j] = self.freq[i].saturating_add(self.freq[i + 1]);
self.son[j] = i;
self.parent[i] = j;
self.parent[i + 1] = j;
i += 2;
j += 1;
}
self.freq[LZH_T] = u16::MAX;
self.parent[LZH_R] = 0;
}
fn decode_char(&mut self) -> Result<usize> {
let mut node = self.son[LZH_R];
while node < LZH_T {
let bit = usize::from(self.bit_reader.read_bit()?);
node = self.son[node + bit];
}
let c = node - LZH_T;
self.update(c);
Ok(c)
}
fn decode_position(&mut self) -> Result<usize> {
let i = self.bit_reader.read_bits(8)? as usize;
let mut c = usize::from(self.d_code[i]) << 6;
let mut j = usize::from(self.d_len[i]).saturating_sub(2);
while j > 0 {
j -= 1;
c |= usize::from(self.bit_reader.read_bit()?) << j;
}
Ok(c | (i & 0x3F))
}
fn update(&mut self, c: usize) {
if self.freq[LZH_R] == LZH_MAX_FREQ {
self.reconstruct();
}
let mut current = self.parent[c + LZH_T];
loop {
self.freq[current] = self.freq[current].saturating_add(1);
let freq = self.freq[current];
if current + 1 < self.freq.len() && freq > self.freq[current + 1] {
let mut swap_idx = current + 1;
while swap_idx + 1 < self.freq.len() && freq > self.freq[swap_idx + 1] {
swap_idx += 1;
}
self.freq.swap(current, swap_idx);
let left = self.son[current];
let right = self.son[swap_idx];
self.son[current] = right;
self.son[swap_idx] = left;
self.parent[left] = swap_idx;
if left < LZH_T {
self.parent[left + 1] = swap_idx;
}
self.parent[right] = current;
if right < LZH_T {
self.parent[right + 1] = current;
}
current = swap_idx;
}
current = self.parent[current];
if current == 0 {
break;
}
}
}
fn reconstruct(&mut self) {
let mut j = 0usize;
for i in 0..LZH_T {
if self.son[i] >= LZH_T {
self.freq[j] = (self.freq[i].saturating_add(1)) / 2;
self.son[j] = self.son[i];
j += 1;
}
}
let mut i = 0usize;
let mut current = LZH_N_CHAR;
while current < LZH_T {
let sum = self.freq[i].saturating_add(self.freq[i + 1]);
self.freq[current] = sum;
let mut insert_at = current;
while insert_at > 0 && sum < self.freq[insert_at - 1] {
insert_at -= 1;
}
for move_idx in (insert_at..current).rev() {
self.freq[move_idx + 1] = self.freq[move_idx];
self.son[move_idx + 1] = self.son[move_idx];
}
self.freq[insert_at] = sum;
self.son[insert_at] = i;
i += 2;
current += 1;
}
for idx in 0..LZH_T {
let node = self.son[idx];
self.parent[node] = idx;
if node < LZH_T {
self.parent[node + 1] = idx;
}
}
self.freq[LZH_T] = u16::MAX;
self.parent[LZH_R] = 0;
}
}
struct BitReader<'a> {
data: &'a [u8],
byte_pos: usize,
bit_mask: u8,
current_byte: u8,
xor_state: Option<XorState>,
}
impl<'a> BitReader<'a> {
fn new(data: &'a [u8], xor_key: Option<u16>) -> Self {
Self {
data,
byte_pos: 0,
bit_mask: 0x80,
current_byte: 0,
xor_state: xor_key.map(XorState::new),
}
}
fn read_bit(&mut self) -> Result<u8> {
if self.bit_mask == 0x80 {
let Some(mut byte) = self.data.get(self.byte_pos).copied() else {
return Err(Error::DecompressionFailed("lzss-huffman: unexpected EOF"));
};
if let Some(state) = &mut self.xor_state {
byte = state.decrypt_byte(byte);
}
self.current_byte = byte;
}
let bit = if (self.current_byte & self.bit_mask) != 0 {
1
} else {
0
};
self.bit_mask >>= 1;
if self.bit_mask == 0 {
self.bit_mask = 0x80;
self.byte_pos = self.byte_pos.saturating_add(1);
}
Ok(bit)
}
fn read_bits(&mut self, bits: usize) -> Result<u32> {
let mut value = 0u32;
for _ in 0..bits {
value = (value << 1) | u32::from(self.read_bit()?);
}
Ok(value)
}
}
-79
View File
@@ -1,79 +0,0 @@
use super::xor::XorState;
use crate::error::Error;
use crate::Result;
/// Simple LZSS decompression with optional on-the-fly XOR decryption
pub fn lzss_decompress_simple(
data: &[u8],
expected_size: usize,
xor_key: Option<u16>,
) -> Result<Vec<u8>> {
let mut ring = [0x20u8; 0x1000];
let mut ring_pos = 0xFEEusize;
let mut out = Vec::with_capacity(expected_size);
let mut in_pos = 0usize;
let mut control = 0u8;
let mut bits_left = 0u8;
// XOR state for on-the-fly decryption
let mut xor_state = xor_key.map(XorState::new);
// Helper to read byte with optional XOR decryption
let read_byte = |pos: usize, state: &mut Option<XorState>| -> Option<u8> {
let encrypted = data.get(pos).copied()?;
Some(if let Some(ref mut s) = state {
s.decrypt_byte(encrypted)
} else {
encrypted
})
};
while out.len() < expected_size {
if bits_left == 0 {
let byte = read_byte(in_pos, &mut xor_state)
.ok_or(Error::DecompressionFailed("lzss-simple: unexpected EOF"))?;
control = byte;
in_pos += 1;
bits_left = 8;
}
if (control & 1) != 0 {
let byte = read_byte(in_pos, &mut xor_state)
.ok_or(Error::DecompressionFailed("lzss-simple: unexpected EOF"))?;
in_pos += 1;
out.push(byte);
ring[ring_pos] = byte;
ring_pos = (ring_pos + 1) & 0x0FFF;
} else {
let low = read_byte(in_pos, &mut xor_state)
.ok_or(Error::DecompressionFailed("lzss-simple: unexpected EOF"))?;
let high = read_byte(in_pos + 1, &mut xor_state)
.ok_or(Error::DecompressionFailed("lzss-simple: unexpected EOF"))?;
in_pos += 2;
let offset = usize::from(low) | (usize::from(high & 0xF0) << 4);
let length = usize::from((high & 0x0F) + 3);
for step in 0..length {
let byte = ring[(offset + step) & 0x0FFF];
out.push(byte);
ring[ring_pos] = byte;
ring_pos = (ring_pos + 1) & 0x0FFF;
if out.len() >= expected_size {
break;
}
}
}
control >>= 1;
bits_left -= 1;
}
if out.len() != expected_size {
return Err(Error::DecompressionFailed("lzss-simple"));
}
Ok(out)
}
-9
View File
@@ -1,9 +0,0 @@
pub mod deflate;
pub mod lzh;
pub mod lzss;
pub mod xor;
pub use deflate::decode_deflate;
pub use lzh::lzss_huffman_decompress;
pub use lzss::lzss_decompress_simple;
pub use xor::{xor_stream, XorState};
-29
View File
@@ -1,29 +0,0 @@
/// XOR cipher state for RsLi format
pub struct XorState {
lo: u8,
hi: u8,
}
impl XorState {
/// Create new XOR state from 16-bit key
pub fn new(key16: u16) -> Self {
Self {
lo: (key16 & 0xFF) as u8,
hi: ((key16 >> 8) & 0xFF) as u8,
}
}
/// Decrypt a single byte and update state
pub fn decrypt_byte(&mut self, encrypted: u8) -> u8 {
self.lo = self.hi ^ self.lo.wrapping_shl(1);
let decrypted = encrypted ^ self.lo;
self.hi = self.lo ^ (self.hi >> 1);
decrypted
}
}
/// Decrypt entire buffer with XOR stream cipher
pub fn xor_stream(data: &[u8], key16: u16) -> Vec<u8> {
let mut state = XorState::new(key16);
data.iter().map(|&b| state.decrypt_byte(b)).collect()
}
-140
View File
@@ -1,140 +0,0 @@
use core::fmt;
#[derive(Debug)]
#[non_exhaustive]
pub enum Error {
Io(std::io::Error),
InvalidMagic {
got: [u8; 2],
},
UnsupportedVersion {
got: u8,
},
InvalidEntryCount {
got: i16,
},
TooManyEntries {
got: usize,
},
EntryTableOutOfBounds {
table_offset: u64,
table_len: u64,
file_len: u64,
},
EntryTableDecryptFailed,
CorruptEntryTable(&'static str),
EntryIdOutOfRange {
id: u32,
entry_count: u32,
},
EntryDataOutOfBounds {
id: u32,
offset: u64,
size: u32,
file_len: u64,
},
AoTrailerInvalid,
MediaOverlayOutOfBounds {
overlay: u32,
file_len: u64,
},
UnsupportedMethod {
raw: u32,
},
PackedSizePastEof {
id: u32,
offset: u64,
packed_size: u32,
file_len: u64,
},
DeflateEofPlusOneQuirkRejected {
id: u32,
},
DecompressionFailed(&'static str),
OutputSizeMismatch {
expected: u32,
got: u32,
},
IntegerOverflow,
}
impl From<std::io::Error> for Error {
fn from(value: std::io::Error) -> Self {
Self::Io(value)
}
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Error::Io(e) => write!(f, "I/O error: {e}"),
Error::InvalidMagic { got } => write!(f, "invalid RsLi magic: {got:02X?}"),
Error::UnsupportedVersion { got } => write!(f, "unsupported RsLi version: {got:#x}"),
Error::InvalidEntryCount { got } => write!(f, "invalid entry_count: {got}"),
Error::TooManyEntries { got } => write!(f, "too many entries: {got} exceeds u32::MAX"),
Error::EntryTableOutOfBounds {
table_offset,
table_len,
file_len,
} => write!(
f,
"entry table out of bounds: off={table_offset}, len={table_len}, file={file_len}"
),
Error::EntryTableDecryptFailed => write!(f, "failed to decrypt entry table"),
Error::CorruptEntryTable(s) => write!(f, "corrupt entry table: {s}"),
Error::EntryIdOutOfRange { id, entry_count } => {
write!(f, "entry id out of range: id={id}, count={entry_count}")
}
Error::EntryDataOutOfBounds {
id,
offset,
size,
file_len,
} => write!(
f,
"entry data out of bounds: id={id}, off={offset}, size={size}, file={file_len}"
),
Error::AoTrailerInvalid => write!(f, "invalid AO trailer"),
Error::MediaOverlayOutOfBounds { overlay, file_len } => {
write!(
f,
"media overlay out of bounds: overlay={overlay}, file={file_len}"
)
}
Error::UnsupportedMethod { raw } => write!(f, "unsupported packing method: {raw:#x}"),
Error::PackedSizePastEof {
id,
offset,
packed_size,
file_len,
} => write!(
f,
"packed range past EOF: id={id}, off={offset}, size={packed_size}, file={file_len}"
),
Error::DeflateEofPlusOneQuirkRejected { id } => {
write!(f, "deflate EOF+1 quirk rejected for entry {id}")
}
Error::DecompressionFailed(s) => write!(f, "decompression failed: {s}"),
Error::OutputSizeMismatch { expected, got } => {
write!(f, "output size mismatch: expected={expected}, got={got}")
}
Error::IntegerOverflow => write!(f, "integer overflow"),
}
}
}
impl std::error::Error for Error {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
Self::Io(err) => Some(err),
_ => None,
}
}
}
-411
View File
@@ -1,411 +0,0 @@
pub mod compress;
pub mod error;
pub mod parse;
use crate::compress::{
decode_deflate, lzss_decompress_simple, lzss_huffman_decompress, xor_stream,
};
use crate::error::Error;
use crate::parse::{c_name_bytes, cmp_c_string, parse_library};
use common::{OutputBuffer, ResourceData};
use std::cmp::Ordering;
use std::fs;
use std::path::Path;
use std::sync::Arc;
pub type Result<T> = core::result::Result<T, Error>;
#[derive(Clone, Debug)]
pub struct OpenOptions {
pub allow_ao_trailer: bool,
pub allow_deflate_eof_plus_one: bool,
}
impl Default for OpenOptions {
fn default() -> Self {
Self {
allow_ao_trailer: true,
allow_deflate_eof_plus_one: true,
}
}
}
#[derive(Debug)]
pub struct Library {
bytes: Arc<[u8]>,
entries: Vec<EntryRecord>,
#[cfg(test)]
pub(crate) header_raw: [u8; 32],
#[cfg(test)]
pub(crate) table_plain_original: Vec<u8>,
#[cfg(test)]
pub(crate) xor_seed: u32,
#[cfg(test)]
pub(crate) source_size: usize,
#[cfg(test)]
pub(crate) trailer_raw: Option<[u8; 6]>,
}
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)]
pub struct EntryId(pub u32);
#[derive(Clone, Debug)]
pub struct EntryMeta {
pub name: String,
pub flags: i32,
pub method: PackMethod,
pub data_offset: u64,
pub packed_size: u32,
pub unpacked_size: u32,
}
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub enum PackMethod {
None,
XorOnly,
Lzss,
XorLzss,
LzssHuffman,
XorLzssHuffman,
Deflate,
Unknown(u32),
}
#[derive(Copy, Clone, Debug)]
pub struct EntryRef<'a> {
pub id: EntryId,
pub meta: &'a EntryMeta,
}
pub struct PackedResource {
pub meta: EntryMeta,
pub packed: Vec<u8>,
}
#[derive(Clone, Debug)]
pub(crate) struct EntryRecord {
pub(crate) meta: EntryMeta,
pub(crate) name_raw: [u8; 12],
pub(crate) sort_to_original: i16,
pub(crate) key16: u16,
#[cfg(test)]
pub(crate) data_offset_raw: u32,
pub(crate) packed_size_declared: u32,
pub(crate) packed_size_available: usize,
pub(crate) effective_offset: usize,
}
impl Library {
pub fn open_path(path: impl AsRef<Path>) -> Result<Self> {
Self::open_path_with(path, OpenOptions::default())
}
pub fn open_path_with(path: impl AsRef<Path>, opts: OpenOptions) -> Result<Self> {
let bytes = fs::read(path.as_ref())?;
let arc: Arc<[u8]> = Arc::from(bytes.into_boxed_slice());
parse_library(arc, opts)
}
pub fn entry_count(&self) -> usize {
self.entries.len()
}
pub fn entries(&self) -> impl Iterator<Item = EntryRef<'_>> {
self.entries
.iter()
.enumerate()
.map(|(idx, entry)| EntryRef {
id: EntryId(u32::try_from(idx).expect("entry count validated at parse")),
meta: &entry.meta,
})
}
pub fn find(&self, name: &str) -> Option<EntryId> {
if self.entries.is_empty() {
return None;
}
const MAX_INLINE_NAME: usize = 12;
// Fast path: use stack allocation for short ASCII names (95% of cases)
if name.len() <= MAX_INLINE_NAME && name.is_ascii() {
let mut buf = [0u8; MAX_INLINE_NAME];
for (i, &b) in name.as_bytes().iter().enumerate() {
buf[i] = b.to_ascii_uppercase();
}
return self.find_impl(&buf[..name.len()]);
}
// Slow path: heap allocation for long or non-ASCII names
let query = name.to_ascii_uppercase();
self.find_impl(query.as_bytes())
}
fn find_impl(&self, query_bytes: &[u8]) -> Option<EntryId> {
// Binary search
let mut low = 0usize;
let mut high = self.entries.len();
while low < high {
let mid = low + (high - low) / 2;
let idx = self.entries[mid].sort_to_original;
if idx < 0 {
break;
}
let idx = usize::try_from(idx).ok()?;
if idx >= self.entries.len() {
break;
}
let cmp = cmp_c_string(query_bytes, c_name_bytes(&self.entries[idx].name_raw));
match cmp {
Ordering::Less => high = mid,
Ordering::Greater => low = mid + 1,
Ordering::Equal => {
return Some(EntryId(
u32::try_from(idx).expect("entry count validated at parse"),
))
}
}
}
// Linear fallback search
self.entries.iter().enumerate().find_map(|(idx, entry)| {
if cmp_c_string(query_bytes, c_name_bytes(&entry.name_raw)) == Ordering::Equal {
Some(EntryId(
u32::try_from(idx).expect("entry count validated at parse"),
))
} else {
None
}
})
}
pub fn get(&self, id: EntryId) -> Option<EntryRef<'_>> {
let idx = usize::try_from(id.0).ok()?;
let entry = self.entries.get(idx)?;
Some(EntryRef {
id,
meta: &entry.meta,
})
}
pub fn load(&self, id: EntryId) -> Result<Vec<u8>> {
let entry = self.entry_by_id(id)?;
let packed = self.packed_slice(id, entry)?;
decode_payload(
packed,
entry.meta.method,
entry.key16,
entry.meta.unpacked_size,
)
}
pub fn load_into(&self, id: EntryId, out: &mut dyn OutputBuffer) -> Result<usize> {
let decoded = self.load(id)?;
out.write_exact(&decoded)?;
Ok(decoded.len())
}
pub fn load_packed(&self, id: EntryId) -> Result<PackedResource> {
let entry = self.entry_by_id(id)?;
let packed = self.packed_slice(id, entry)?.to_vec();
Ok(PackedResource {
meta: entry.meta.clone(),
packed,
})
}
pub fn unpack(&self, packed: &PackedResource) -> Result<Vec<u8>> {
let key16 = self.resolve_key_for_meta(&packed.meta).unwrap_or(0);
let method = packed.meta.method;
if needs_xor_key(method) && self.resolve_key_for_meta(&packed.meta).is_none() {
return Err(Error::CorruptEntryTable(
"cannot resolve XOR key for packed resource",
));
}
decode_payload(&packed.packed, method, key16, packed.meta.unpacked_size)
}
pub fn load_fast(&self, id: EntryId) -> Result<ResourceData<'_>> {
let entry = self.entry_by_id(id)?;
if entry.meta.method == PackMethod::None {
let packed = self.packed_slice(id, entry)?;
let size =
usize::try_from(entry.meta.unpacked_size).map_err(|_| Error::IntegerOverflow)?;
if packed.len() < size {
return Err(Error::OutputSizeMismatch {
expected: entry.meta.unpacked_size,
got: u32::try_from(packed.len()).unwrap_or(u32::MAX),
});
}
return Ok(ResourceData::Borrowed(&packed[..size]));
}
Ok(ResourceData::Owned(self.load(id)?))
}
fn entry_by_id(&self, id: EntryId) -> Result<&EntryRecord> {
let idx = usize::try_from(id.0).map_err(|_| Error::IntegerOverflow)?;
self.entries
.get(idx)
.ok_or_else(|| Error::EntryIdOutOfRange {
id: id.0,
entry_count: self.entries.len().try_into().unwrap_or(u32::MAX),
})
}
fn packed_slice<'a>(&'a self, id: EntryId, entry: &EntryRecord) -> Result<&'a [u8]> {
let start = entry.effective_offset;
let end = start
.checked_add(entry.packed_size_available)
.ok_or(Error::IntegerOverflow)?;
self.bytes
.get(start..end)
.ok_or(Error::EntryDataOutOfBounds {
id: id.0,
offset: u64::try_from(start).unwrap_or(u64::MAX),
size: entry.packed_size_declared,
file_len: u64::try_from(self.bytes.len()).unwrap_or(u64::MAX),
})
}
fn resolve_key_for_meta(&self, meta: &EntryMeta) -> Option<u16> {
self.entries
.iter()
.find(|entry| {
entry.meta.name == meta.name
&& entry.meta.flags == meta.flags
&& entry.meta.data_offset == meta.data_offset
&& entry.meta.packed_size == meta.packed_size
&& entry.meta.unpacked_size == meta.unpacked_size
&& entry.meta.method == meta.method
})
.map(|entry| entry.key16)
}
#[cfg(test)]
pub(crate) fn rebuild_from_parsed_metadata(&self) -> Result<Vec<u8>> {
let trailer_len = usize::from(self.trailer_raw.is_some()) * 6;
let pre_trailer_size = self
.source_size
.checked_sub(trailer_len)
.ok_or(Error::IntegerOverflow)?;
let count = self.entries.len();
let table_len = count.checked_mul(32).ok_or(Error::IntegerOverflow)?;
let table_end = 32usize
.checked_add(table_len)
.ok_or(Error::IntegerOverflow)?;
if pre_trailer_size < table_end {
return Err(Error::EntryTableOutOfBounds {
table_offset: 32,
table_len: u64::try_from(table_len).map_err(|_| Error::IntegerOverflow)?,
file_len: u64::try_from(pre_trailer_size).map_err(|_| Error::IntegerOverflow)?,
});
}
let mut out = vec![0u8; pre_trailer_size];
out[0..32].copy_from_slice(&self.header_raw);
let encrypted_table =
xor_stream(&self.table_plain_original, (self.xor_seed & 0xFFFF) as u16);
out[32..table_end].copy_from_slice(&encrypted_table);
let mut occupied = vec![false; pre_trailer_size];
for byte in occupied.iter_mut().take(table_end) {
*byte = true;
}
for (idx, entry) in self.entries.iter().enumerate() {
let packed = self
.load_packed(EntryId(
u32::try_from(idx).expect("entry count validated at parse"),
))?
.packed;
let start =
usize::try_from(entry.data_offset_raw).map_err(|_| Error::IntegerOverflow)?;
for (offset, byte) in packed.iter().copied().enumerate() {
let pos = start.checked_add(offset).ok_or(Error::IntegerOverflow)?;
if pos >= out.len() {
return Err(Error::PackedSizePastEof {
id: u32::try_from(idx).expect("entry count validated at parse"),
offset: u64::from(entry.data_offset_raw),
packed_size: entry.packed_size_declared,
file_len: u64::try_from(out.len()).map_err(|_| Error::IntegerOverflow)?,
});
}
if occupied[pos] && out[pos] != byte {
return Err(Error::CorruptEntryTable("packed payload overlap conflict"));
}
out[pos] = byte;
occupied[pos] = true;
}
}
if let Some(trailer) = self.trailer_raw {
out.extend_from_slice(&trailer);
}
Ok(out)
}
}
fn decode_payload(
packed: &[u8],
method: PackMethod,
key16: u16,
unpacked_size: u32,
) -> Result<Vec<u8>> {
let expected = usize::try_from(unpacked_size).map_err(|_| Error::IntegerOverflow)?;
let out = match method {
PackMethod::None => {
if packed.len() < expected {
return Err(Error::OutputSizeMismatch {
expected: unpacked_size,
got: u32::try_from(packed.len()).unwrap_or(u32::MAX),
});
}
packed[..expected].to_vec()
}
PackMethod::XorOnly => {
if packed.len() < expected {
return Err(Error::OutputSizeMismatch {
expected: unpacked_size,
got: u32::try_from(packed.len()).unwrap_or(u32::MAX),
});
}
xor_stream(&packed[..expected], key16)
}
PackMethod::Lzss => lzss_decompress_simple(packed, expected, None)?,
PackMethod::XorLzss => {
// Optimized: XOR on-the-fly during decompression instead of creating temp buffer
lzss_decompress_simple(packed, expected, Some(key16))?
}
PackMethod::LzssHuffman => lzss_huffman_decompress(packed, expected, None)?,
PackMethod::XorLzssHuffman => {
// Optimized: XOR on-the-fly during decompression
lzss_huffman_decompress(packed, expected, Some(key16))?
}
PackMethod::Deflate => decode_deflate(packed)?,
PackMethod::Unknown(raw) => return Err(Error::UnsupportedMethod { raw }),
};
if out.len() != expected {
return Err(Error::OutputSizeMismatch {
expected: unpacked_size,
got: u32::try_from(out.len()).unwrap_or(u32::MAX),
});
}
Ok(out)
}
fn needs_xor_key(method: PackMethod) -> bool {
matches!(
method,
PackMethod::XorOnly | PackMethod::XorLzss | PackMethod::XorLzssHuffman
)
}
#[cfg(test)]
mod tests;
-267
View File
@@ -1,267 +0,0 @@
use crate::compress::xor::xor_stream;
use crate::error::Error;
use crate::{EntryMeta, EntryRecord, Library, OpenOptions, PackMethod, Result};
use std::cmp::Ordering;
use std::sync::Arc;
pub fn parse_library(bytes: Arc<[u8]>, opts: OpenOptions) -> Result<Library> {
if bytes.len() < 32 {
return Err(Error::EntryTableOutOfBounds {
table_offset: 32,
table_len: 0,
file_len: u64::try_from(bytes.len()).map_err(|_| Error::IntegerOverflow)?,
});
}
let mut header_raw = [0u8; 32];
header_raw.copy_from_slice(&bytes[0..32]);
if &bytes[0..2] != b"NL" {
let mut got = [0u8; 2];
got.copy_from_slice(&bytes[0..2]);
return Err(Error::InvalidMagic { got });
}
if bytes[3] != 0x01 {
return Err(Error::UnsupportedVersion { got: bytes[3] });
}
let entry_count = i16::from_le_bytes([bytes[4], bytes[5]]);
if entry_count < 0 {
return Err(Error::InvalidEntryCount { got: entry_count });
}
let count = usize::try_from(entry_count).map_err(|_| Error::IntegerOverflow)?;
// Validate entry_count fits in u32 (required for EntryId)
if count > u32::MAX as usize {
return Err(Error::TooManyEntries { got: count });
}
let xor_seed = u32::from_le_bytes([bytes[20], bytes[21], bytes[22], bytes[23]]);
let table_len = count.checked_mul(32).ok_or(Error::IntegerOverflow)?;
let table_offset = 32usize;
let table_end = table_offset
.checked_add(table_len)
.ok_or(Error::IntegerOverflow)?;
if table_end > bytes.len() {
return Err(Error::EntryTableOutOfBounds {
table_offset: u64::try_from(table_offset).map_err(|_| Error::IntegerOverflow)?,
table_len: u64::try_from(table_len).map_err(|_| Error::IntegerOverflow)?,
file_len: u64::try_from(bytes.len()).map_err(|_| Error::IntegerOverflow)?,
});
}
let table_enc = &bytes[table_offset..table_end];
let table_plain_original = xor_stream(table_enc, (xor_seed & 0xFFFF) as u16);
if table_plain_original.len() != table_len {
return Err(Error::EntryTableDecryptFailed);
}
let (overlay, trailer_raw) = parse_ao_trailer(&bytes, opts.allow_ao_trailer)?;
#[cfg(not(test))]
let _ = trailer_raw;
let mut entries = Vec::with_capacity(count);
for idx in 0..count {
let row = &table_plain_original[idx * 32..(idx + 1) * 32];
let mut name_raw = [0u8; 12];
name_raw.copy_from_slice(&row[0..12]);
let flags_signed = i16::from_le_bytes([row[16], row[17]]);
let sort_to_original = i16::from_le_bytes([row[18], row[19]]);
let unpacked_size = u32::from_le_bytes([row[20], row[21], row[22], row[23]]);
let data_offset_raw = u32::from_le_bytes([row[24], row[25], row[26], row[27]]);
let packed_size_declared = u32::from_le_bytes([row[28], row[29], row[30], row[31]]);
let method_raw = (flags_signed as u16 as u32) & 0x1E0;
let method = parse_method(method_raw);
let effective_offset_u64 = u64::from(data_offset_raw)
.checked_add(u64::from(overlay))
.ok_or(Error::IntegerOverflow)?;
let effective_offset =
usize::try_from(effective_offset_u64).map_err(|_| Error::IntegerOverflow)?;
let packed_size_usize =
usize::try_from(packed_size_declared).map_err(|_| Error::IntegerOverflow)?;
let mut packed_size_available = packed_size_usize;
let end = effective_offset_u64
.checked_add(u64::from(packed_size_declared))
.ok_or(Error::IntegerOverflow)?;
let file_len_u64 = u64::try_from(bytes.len()).map_err(|_| Error::IntegerOverflow)?;
if end > file_len_u64 {
if method_raw == 0x100 && end == file_len_u64 + 1 {
if opts.allow_deflate_eof_plus_one {
packed_size_available = packed_size_available
.checked_sub(1)
.ok_or(Error::IntegerOverflow)?;
} else {
return Err(Error::DeflateEofPlusOneQuirkRejected {
id: u32::try_from(idx).expect("entry count validated at parse"),
});
}
} else {
return Err(Error::PackedSizePastEof {
id: u32::try_from(idx).expect("entry count validated at parse"),
offset: effective_offset_u64,
packed_size: packed_size_declared,
file_len: file_len_u64,
});
}
}
let available_end = effective_offset
.checked_add(packed_size_available)
.ok_or(Error::IntegerOverflow)?;
if available_end > bytes.len() {
return Err(Error::EntryDataOutOfBounds {
id: u32::try_from(idx).expect("entry count validated at parse"),
offset: effective_offset_u64,
size: packed_size_declared,
file_len: file_len_u64,
});
}
let name = decode_name(c_name_bytes(&name_raw));
entries.push(EntryRecord {
meta: EntryMeta {
name,
flags: i32::from(flags_signed),
method,
data_offset: effective_offset_u64,
packed_size: packed_size_declared,
unpacked_size,
},
name_raw,
sort_to_original,
key16: sort_to_original as u16,
#[cfg(test)]
data_offset_raw,
packed_size_declared,
packed_size_available,
effective_offset,
});
}
let presorted_flag = u16::from_le_bytes([bytes[14], bytes[15]]);
if presorted_flag == 0xABBA {
let mut seen = vec![false; count];
for entry in &entries {
let idx = i32::from(entry.sort_to_original);
if idx < 0 {
return Err(Error::CorruptEntryTable(
"sort_to_original is not a valid permutation index",
));
}
let idx = usize::try_from(idx).map_err(|_| Error::IntegerOverflow)?;
if idx >= count {
return Err(Error::CorruptEntryTable(
"sort_to_original is not a valid permutation index",
));
}
if seen[idx] {
return Err(Error::CorruptEntryTable(
"sort_to_original is not a permutation",
));
}
seen[idx] = true;
}
if seen.iter().any(|value| !*value) {
return Err(Error::CorruptEntryTable(
"sort_to_original is not a permutation",
));
}
} else {
let mut sorted: Vec<usize> = (0..count).collect();
sorted.sort_by(|a, b| {
cmp_c_string(
c_name_bytes(&entries[*a].name_raw),
c_name_bytes(&entries[*b].name_raw),
)
});
for (idx, entry) in entries.iter_mut().enumerate() {
entry.sort_to_original =
i16::try_from(sorted[idx]).map_err(|_| Error::IntegerOverflow)?;
entry.key16 = entry.sort_to_original as u16;
}
}
#[cfg(test)]
let source_size = bytes.len();
Ok(Library {
bytes,
entries,
#[cfg(test)]
header_raw,
#[cfg(test)]
table_plain_original,
#[cfg(test)]
xor_seed,
#[cfg(test)]
source_size,
#[cfg(test)]
trailer_raw,
})
}
fn parse_ao_trailer(bytes: &[u8], allow: bool) -> Result<(u32, Option<[u8; 6]>)> {
if !allow || bytes.len() < 6 {
return Ok((0, None));
}
if &bytes[bytes.len() - 6..bytes.len() - 4] != b"AO" {
return Ok((0, None));
}
let mut trailer = [0u8; 6];
trailer.copy_from_slice(&bytes[bytes.len() - 6..]);
let overlay = u32::from_le_bytes([trailer[2], trailer[3], trailer[4], trailer[5]]);
if u64::from(overlay) > u64::try_from(bytes.len()).map_err(|_| Error::IntegerOverflow)? {
return Err(Error::MediaOverlayOutOfBounds {
overlay,
file_len: u64::try_from(bytes.len()).map_err(|_| Error::IntegerOverflow)?,
});
}
Ok((overlay, Some(trailer)))
}
pub fn parse_method(raw: u32) -> PackMethod {
match raw {
0x000 => PackMethod::None,
0x020 => PackMethod::XorOnly,
0x040 => PackMethod::Lzss,
0x060 => PackMethod::XorLzss,
0x080 => PackMethod::LzssHuffman,
0x0A0 => PackMethod::XorLzssHuffman,
0x100 => PackMethod::Deflate,
other => PackMethod::Unknown(other),
}
}
fn decode_name(name: &[u8]) -> String {
name.iter().map(|b| char::from(*b)).collect()
}
pub fn c_name_bytes(raw: &[u8; 12]) -> &[u8] {
let len = raw.iter().position(|&b| b == 0).unwrap_or(raw.len());
&raw[..len]
}
pub fn cmp_c_string(a: &[u8], b: &[u8]) -> Ordering {
let min_len = a.len().min(b.len());
let mut idx = 0usize;
while idx < min_len {
if a[idx] != b[idx] {
return a[idx].cmp(&b[idx]);
}
idx += 1;
}
a.len().cmp(&b.len())
}
File diff suppressed because it is too large Load Diff
+200
View File
@@ -0,0 +1,200 @@
# Глоссарий
Глоссарий объясняет термины в том смысле, в котором они используются в этой
книге. Короткое определение не заменяет профильную главу: практический контракт
понятия раскрывается в соответствующем томе или справочной странице.
## Бинарные файлы и ABI
**PE (Portable Executable)** -- формат исполняемых файлов Windows: EXE и DLL.
Он содержит заголовки, секции, таблицы импортов и экспортов, relocations и
адрес точки входа.
**Image base** -- предпочтительный адрес начала загруженного PE-образа.
**VA** -- виртуальный адрес в процессе. **RVA** -- адрес относительно image
base.
**Import** -- внешняя функция или переменная, которую модуль получает из другой
DLL. **Export** -- символ, предоставляемый другим модулям. Имя, ordinal и
calling convention вместе образуют часть binary contract.
**ABI** -- соглашение о двоичном взаимодействии: размещение аргументов, возврат
значений, очистка stack, layout структур, порядок virtual methods и правила
владения.
**Calling convention** -- часть ABI, определяющая передачу аргументов и очистку
stack. Для исследованного 32-bit code важны `__cdecl`, `__stdcall` и
`__thiscall`.
**Vtable** -- массив указателей на virtual methods C++-объекта. Запись
`vtable +0x34` означает вызов указателя по байтовому смещению `0x34` от начала
таблицы.
**Static analysis** исследует файл без исполнения: disassembly, strings,
imports, call graph и data flow. **Dynamic analysis** наблюдает работающую
программу: breakpoints, traces, API hooks, memory state и packet/frame captures.
**Evidence** -- повторяемое наблюдение. **Inference** -- вывод, объединяющий
несколько наблюдений. **Hypothesis** -- рабочее предположение, ещё не
подтверждённое достаточным экспериментом.
## Форматы данных
**Archive** -- контейнер, объединяющий множество ресурсов. **Entry** -- запись
его каталога. **Payload** -- полезные bytes конкретной записи.
**Magic** -- короткая сигнатура формата, например `NRes` или `Texm`.
**Version** -- номер варианта layout. Проверка одной magic без проверки version
и размеров недостаточна.
**Offset** -- положение данных относительно начала файла или структуры.
**Size** -- число bytes. **Stride** -- размер одного элемента массива.
**Alignment** -- требование начинать данные на offset, кратном заданному числу.
**Little-endian** -- порядок, в котором младший byte многобайтного числа
расположен первым. Основные числовые поля форматов Iron3D используют этот
порядок.
**Fixed-size string** -- поле заранее известной длины. Полезная строка
заканчивается первым NUL, но оставшиеся bytes могут содержать служебный хвост и
должны сохраняться.
**Opaque field** -- поле с доказанными offset и size, но не установленным
предметным смыслом. Его безопасно читать и копировать, но нельзя очищать или
переосмысливать без эксперимента.
**Invariant** -- условие, которое обязано выполняться: range лежит внутри
payload, индекс указывает на существующий элемент, count соответствует размеру
секции.
**Strict reader** отклоняет любое нарушение контракта. **Compatibility reader**
дополнительно воспроизводит только известные особенности оригинала.
**Fallback** -- явно предписанный запасной путь, например material `DEFAULT`,
затем entry 0. **Heuristic** -- догадка по похожим данным; она не должна
незаметно заменять доказанный fallback.
**Roundtrip** -- последовательность decode -> encode. **Byte-identical
roundtrip** создаёт файл, полностью совпадающий с исходным. **Lossless editor**
может изменить известное поле, сохранив все остальные bytes и порядок записей.
## Ресурсы
**NRes** -- основной контейнер ресурсов с каталогом в конце файла.
**RsLi** -- библиотечный архив с каталогом в начале файла и несколькими методами
упаковки payload.
**TMA** -- mission data: paths, clans, placed objects, properties, land path и
extras.
**MSH** -- модель Iron3D, представленная как NRes с entries для geometry,
nodes, slots, batches, animation и auxiliary streams.
**WEAR** -- таблица внешнего вида модели, переводящая material index в MAT0
name и lightmap slots.
**MAT0** -- материал: phases, parameters, animation blocks и texture references.
**Texm** -- texture payload с header, palette, mip chain и optional Page atlas.
**FXID** -- ресурс эффектов: команды, references, lifetime, random/time modes и
runtime instances.
## Игровой runtime
**Engine** -- программная среда, которая загружает данные, ведёт время,
исполняет мир и формирует изображение/звук. **Game** -- правила, миссии и
content поверх engine services.
**World** -- долгоживущее состояние миссии: objects, terrain, время, кланы и
managers. **Scene** -- представление части мира для конкретной обработки,
обычно текущей камеры.
**Game object** -- сущность с идентичностью, transform, properties и lifecycle.
**Component/controller** -- специализированная часть поведения: animation,
physics, AI или rendering representation.
**Simulation** отвечает за изменение мира. **Tick** -- один расчётный шаг.
**Frame** -- одно подготовленное изображение. Число ticks и frames за единицу
времени не обязано совпадать.
**Event/message** -- типизированное сообщение между objects или subsystems.
**Queue traversal** -- стабильный обход зарегистрированных объектов.
**Deferred deletion** -- перенос фактического удаления до безопасной границы.
**Snapshot** -- согласованное состояние, которое renderer читает без изменения
simulation. **Determinism** -- одинаковый результат при одинаковом initial
state, input, времени и порядке событий.
**Authority** -- subsystem или network peer, которому разрешено окончательно
менять состояние объекта. **Mirror object** -- локальное представление объекта,
authority которого находится у другого player.
## Геометрия и рендеринг
**Vertex** -- вершина geometry. **Index** -- номер вершины. **Triangle** --
примитив из трёх индексов.
**Node** -- элемент hierarchy модели со своим local transform. **Slot** в MSH
-- выбранная геометрическая группа для комбинации node, LOD и group. **Batch**
-- непрерывный индексный диапазон с material slot и render state.
**Transform** переводит данные между coordinate spaces. **Matrix** задаёт
линейное преобразование и translation. Порядок умножения matrices является
частью контракта.
**Bounds** -- упрощённый объём для быстрых тестов. **AABB** -- min/max по осям.
**Bounding sphere** -- center и radius.
**Renderer** преобразует подготовленную сцену в изображение. **Backend** --
реализация поверх конкретного API или устройства.
**Draw call** -- команда нарисовать диапазон primitives. **Indexed draw**
использует index buffer и base vertex.
**Material phase** -- одно временное состояние анимированного материала.
**Texture** -- двумерный массив texels. **Mip chain** -- последовательность
уменьшенных уровней texture. **Atlas** -- texture с несколькими под-
изображениями.
**Fixed-function pipeline** -- старый graphics pipeline, где приложение
выбирает predefined transform, lighting, texture-stage и blend states вместо
пользовательских shaders.
**Depth test**, **culling**, **alpha test** и **blending** -- render states,
которые влияют на порядок и видимость fragments.
**Pixel parity** -- совпадение конечного изображения при фиксированных camera,
time, seed, resolution и device profile.
## Навигация, звук и сеть
**Areal** -- логическая область карты с границей, class/flags и связями с
соседями. **Areal graph** -- граф областей и переходов. **Cell grid** --
пространственный индекс для быстрых candidate queries.
**Pathfinding** -- поиск маршрута по graph. **Corridor** -- локальная полоса,
построенная из последовательности areals. **Local steering** корректирует
ближайший шаг внутри corridor.
**Collision proxy** -- упрощённое представление объекта для столкновений.
**Broad phase** быстро находит потенциальные пары. **Narrow phase** выполняет
точную проверку и вычисляет contact.
**Sample** -- декодированные звуковые данные. **Source** -- конкретный
экземпляр воспроизведения с position, gain, loop state и временем. **Listener**
-- положение и ориентация слушателя для 3D spatialization.
**Transport** -- механизм доставки bytes между peers. **Protocol** -- framing,
message types, порядок и правила подтверждения. **Wire compatibility** --
способность обмениваться данными с оригинальным клиентом.
**Serialization** -- преобразование typed state в byte sequence. **Framing** --
способ отделить одно сообщение от следующего. **Reliable delivery** гарантирует
доставку/порядок в пределах выбранной модели; **unreliable delivery** допускает
потери ради задержки.
**Player ID** транспорта и **game player number** -- разные идентичности.
**Ownership transfer** меняет authority объекта. **Replication** передаёт
состояние или события remote mirrors.
+160
View File
@@ -0,0 +1,160 @@
# Границы знания
Этот раздел перечисляет области, где контракт ещё не закрыт полностью. Они не
мешают безопасному чтению и lossless сохранению, но не должны превращаться в
authoring API без динамического подтверждения.
## Render state
Доказаны frame boundaries, world traversal, material resolve и крупные проходы.
Не доказаны символами точные имена renderer vtable slots, полный набор CShade
state transitions и окончательный порядок части transparent/FX/shadow subpasses.
Закрывающий эксперимент: запустить оригинал в совместимой Windows/DirectX
среде, перехватить DirectDraw/Direct3D calls и surface flips, сохранить state
log на минимальных сценах с одним типом материала.
## FXID field-level semantics
Размеры команд, resource references, lifecycle, flags families и используемые
time modes известны. Не закрыто значение каждого поля body opcodes 1--10,
отсутствующий во всех проверенных каталогах opcode 6 и точные формулы редких
time modes.
Закрывающий эксперимент: изменять по одному полю копии эффекта, воспроизводить
его в контролируемой сцене и логировать runtime command object, emitted
primitives, sound events и reads в `Effect.dll`.
## Script VM
Доступны packages, symbols, event sections, variable declarations и version
checks. Полная instruction grammar `.scr`, semantics opcodes и serialization
state ещё не восстановлены.
Закрывающий эксперимент: найти dispatcher loop в `ai.dll`, сопоставить jump
table с instruction sizes, построить disassembler и сравнить выполнение
коротких scripts с оригиналом.
## Saves and campaign state
Найдены `saveslots.cfg` и `missions/dispatcher.ini`, но binary savegame payload,
serialization World3D/AI/script/RNG и migration rules не закрыты.
Нужны сохранения оригинала в контролируемых состояниях: старт миссии, изменение
позиции, здоровья, order/path, FX/timer, script variable, research/economy,
mission completion, pause и non-default game time.
## Physical/control formats
CTLD и связанные resources структурно читаются, count patterns и variants
известны. Не названы все секции, shape types, coefficients и точный contact
solver. То же относится к редким MSH auxiliary streams и части CTPT/NDPR flags.
Закрывающий эксперимент: трассировать `LoadControlSystem`,
`LoadPhysicalModel`, `CreateCollManager` и создание collision objects; связать
каждый изменяемый field с созданным shape, contact или реакцией на движение.
## DirectPlay wire
DirectPlay lifecycle и имена игровых messages известны. Wire framing, payload
schema, reliability flags и `netZipData` требуют записи обмена двух
оригинальных клиентов.
Native interoperability подтверждается только успешным обменом original client
<-> compatibility implementation в обе стороны.
## Shell, HUD, шрифты и локализация
Граница shell подтверждена exports `createShell/getIShell`, `IGUIServer`,
верхнеуровневым UI-pass и файлами `ui/*.cfg`, `DATA/TextRes.cfg`,
`gamefont.rlb` и `sprites.lib`. RsLi framing библиотек закрыт, но widget tree,
layout rules, glyph metrics, sprite command semantics, focus/navigation и HUD
state machine пока не восстановлены до field-level спецификации.
Закрывающий эксперимент: трассировать загрузку `shell_ctrls.cfg`,
`menu_resources.cfg`, `cursor.cfg`, `game_resources.cfg` и `hq.cfg`, сопоставить
GUI object factories и снять command/event captures для меню, HUD, briefing и
диалогов.
## Research, economy and properties
Экспорты `LoadResearch`, `CalcFullResearchCost`, TRF/preload resources и TMA
properties доказывают отдельный слой исследований, стоимости, добычи и
производственных параметров. Формулы стоимости, dependency graph технологий,
inventory/economy transitions и точная типизация всех 16-byte property values
не закрыты.
Закрывающий эксперимент: сопоставить research functions с ресурсами и UI,
снять изменения state на контролируемых покупках/исследованиях и построить
typed schema свойств по consumers, а не по одному имени.
## Rare branches
- `Land.map poly_count > 0`;
- RsLi adaptive methods `0x080` и `0x0A0`;
- Texm formats 556 и 88;
- FX opcode 6;
- редкие material flags и MSH auxiliary streams.
Такие ветки реализуются по бинарному коду и synthetic tests, а статус
corpus-verified получают только после реального файла или runtime trace.
## Dynamic-stage requirements
Оставшиеся вопросы нельзя закрыть только статическими архивами. Нужна
изолированная 32-bit Windows-среда, неизменённые игровые каталоги, manifest
SHA-256, debugger, API/vtable hooks, controlled clocks/input и автоматический
launcher, который восстанавливает snapshot, запускает один test case, собирает
логи и завершает процесс без ручного вмешательства.
Для каждого capture сохраняются build profile, module hashes, mission/resource
key, configuration, device profile, initial state, input/time script и версии
инструментов.
## Local evidence requests
На текущем рабочем месте закрыты статические, corpus и headless runtime gates.
Для macOS Desktop GL есть только безопасный command/state trace и исторический
одноразовый offscreen pixel probe:
- `cargo test -p fparkan-render-gl --offline desktop_gl33_triangle_command_capture`;
- `fixtures/acceptance/macos-gl33-triangle-capture.json`.
`S3-GL-001` не считается закрытым: временный `rustc` probe создал CGL/OpenGL
offscreen FBO, выполнил shader-based triangle draw, прочитал RGBA pixels и
сохранил hash capture, но постоянный workspace adapter по-прежнему не создаёт
SDL window, GL context, GPU resources, shader programs, draw calls или present.
Probe не добавляет project-owned `unsafe` в workspace и остаётся только external
evidence request artifact.
Для повышения `S3-GL-001` до `covered` нужен постоянный macOS backend через
выбранную safe facade stack: SDL event/window/context lifecycle, Desktop GL 3.3
shader/buffer/texture/draw/present path, hidden-window/offscreen smoke test и
licensed local model/terrain frame capture.
Для повышения `S3-GL-002` до `covered` всё ещё нужен воспроизводимый GLES2
backend profile: GLES2 должен создать кадр, сохранить pixel capture и тот же
command/state trace. Локальный Docker probe существующего Rust image не нашёл
`libGL`, `libEGL`, `libGLES` или `libOSMesa`, поэтому закрытие этого gate требует
отдельно предоставленного Docker image с Rust + Mesa/EGL/OSMesa либо разрешения
на установку соответствующего проверочного окружения.
Для текущей macOS-focused цели `S3-GL-002`, `L3-DEVICE-001` и `L5-RG40-001`
помечены как `omitted`: они остаются требованиями portable target scope, но не
блокируют локальный macOS acceptance-аудит. При возврате RG40XX/GLES2 в область
цели эти gates снова должны требовать внешнего evidence.
`L3-DEVICE-001` и `L5-RG40-001` не закрываются локально без RG40XX H или
эквивалентного удалённого runner-а. Требуемое доказательство: запуск выбранной
миссии при 640x480 на целевом профиле, сохранённые stdout/stderr, build
fingerprint, manifest игрового каталога, frame/tick budget, memory budget и
итоговый pass/fail report. Desktop/headless результаты не считаются заменой
on-device smoke.
## Closure criteria
Вопрос считается закрытым только при наличии build fingerprint, raw trace,
parser trace-а, минимального воспроизводимого input/resource/save/message,
формального контракта или явно ограниченной гипотезы, differential test для
изменённых DLL, обновления тематической главы и regression case, запускаемого
без ручного анализа.
+13
View File
@@ -0,0 +1,13 @@
# Current Project Audit
Baseline command:
```text
env RUSTC=/Users/valentineus/.rustup/toolchains/stable-aarch64-apple-darwin/bin/rustc /opt/homebrew/bin/rustup run stable cargo test --workspace --offline
```
Result on 2026-06-22:
- library and binary unit tests compile and pass after aligning SDL2 versions and pinning `toml` to cached `0.8`;
- doctests fail in this shell because `rustdoc` is not in PATH unless `RUSTDOC` is also set to the real toolchain binary;
- full online dependency resolution is unavailable in the sandbox.
+46 -12
View File
@@ -1,17 +1,51 @@
# Welcome to MkDocs
# FParkan
For full documentation visit [mkdocs.org](https://www.mkdocs.org).
FParkan -- самостоятельная техническая книга о восстановлении игрового движка
Iron3D из *Parkan: Iron Strategy*. Она ведёт от запуска оригинальной программы
и карты DLL к форматам ресурсов, загрузке миссии, геометрии, материалам,
рендеру, поведению, звуку, сети и плану чистой совместимой реализации.
## Commands
Сайт оформлен как онлайн-книга: тома читаются последовательно, а справочник
используется как быстрый доступ к форматам, проверочным правилам и границам
доказанного знания.
* `mkdocs new [dir-name]` - Create a new project.
* `mkdocs serve` - Start the live-reloading docs server.
* `mkdocs build` - Build the documentation site.
* `mkdocs -h` - Print help message and exit.
## Как читать
## Project layout
Если вы впервые разбираете игровой движок, начните с тома I и II. Там вводится
лексика, доказательная политика, модульная архитектура и жизненный цикл кадра.
mkdocs.yml # The configuration file.
docs/
index.md # The documentation homepage.
... # Other markdown pages, images and other files.
Если нужна реализация совместимого движка, читайте тома III--VII линейно:
ресурсы, миссии, мир, рендер, интерактивные подсистемы и порядок работ.
Если вы проверяете выводы, переходите к тому VIII и приложениям. Там собраны
уровни уверенности, corpus gates, открытые вопросы и критерии закрытия.
## Восемь томов
1. **Путеводитель и методика** -- назначение книги, маршруты чтения, язык
предметной области и правила проверки.
2. **Запуск, архитектура и игровой цикл** -- `iron_3d.exe`, пятнадцать DLL,
сервисы, World3D, очередь объектов и границы кадра.
3. **Ресурсная система и форматы** -- NRes, RsLi, кэши, имена, `objects.rlb`,
unit DAT и сквозное разрешение ресурсов.
4. **Мир, миссии и runtime** -- TMA, ландшафт, ареалы, маршруты, создание мира
и свойства размещённых объектов.
5. **Геометрия, материалы и рендер** -- MSH, анимация, WEAR, MAT0, Texm, FXID,
свет, атмосфера и полный render frame.
6. **Поведение, управление, звук и сеть** -- AI, Behavior, Wizard, Control,
ввод, камера, звук и DirectPlay-слой.
7. **Руководство по полной реализации** -- целевая архитектура, этапы работ,
тестовый контур, точность, скорость и критерий совместимости.
8. **Справочник и доказательная база** -- ABI, конфигурация, статистика
корпусов, границы знания и глоссарий.
## Политика доказательств
Специфические утверждения об Iron3D принимаются только после локальной проверки
на исполняемых файлах, DLL, демоверсии, полных каталогах Частей 1 и 2 или на
взаимных инвариантах реальных ресурсов. Внешние описания и текущий код FParkan
могут подсказывать вопросы, но не заменяют проверку.
Неизвестные поля не получают правдоподобных имён. Пока смысл не закрыт,
документация фиксирует raw layout, границы, безопасное чтение и lossless
сохранение.
+69
View File
@@ -0,0 +1,69 @@
# WEAR и MAT0
MSH batch хранит только `material_index`. WEAR переводит этот индекс в имя
материала, а MAT0 по этому имени описывает phases, parameters и texture
references.
```text
Batch20.material_index
-> WEAR row
-> MAT0 entry
-> active phase
-> textureName
```
## WEAR
WEAR -- текстовый ресурс type ID `0x52414557`, обычно `*.wea` рядом с моделью.
```text
<wearCount>
<legacyId> <materialName>
...
[empty line]
[LIGHTMAPS
<lightmapCount>
<legacyId> <lightmapName>
...]
```
`legacyId` сохраняется, но выбор выполняется по позиции строки и имени. Между
основной таблицей и `LIGHTMAPS` нужен пустой разделитель.
## MAT0
MAT0 имеет type ID `0x3054414D`, обычно расположен в `Material.lib`. `attr1`
содержит runtime flags, `attr2` -- версию payload.
```c
#pragma pack(push, 1)
struct Mat0PrefixV4Plus {
uint16_t phase_count;
uint16_t animation_block_count;
uint8_t metadata_a;
uint8_t metadata_b;
uint32_t metadata_c_raw;
uint32_t metadata_d_raw;
};
struct Phase34 {
uint8_t parameters[18];
char texture_name[16];
};
#pragma pack(pop)
```
Versioned fields читаются только если версия их содержит. Для старых версий
используются runtime defaults, а raw values сохраняются.
## Fallback
Material resolve:
1. имя из WEAR;
2. `DEFAULT`;
3. entry с индексом 0.
Пустое texture name означает намеренно нетекстурированную поверхность. Lightmap
fallback отдельный: отсутствующий lightmap даёт slot `-1`.
+82
View File
@@ -0,0 +1,82 @@
# MSH
Файл `*.msh` является NRes-контейнером. Geometry, узлы, slots, batches,
animation и служебные streams лежат в entries с разными `type_id`.
## Entry map
```text
type 1 nodes and slot selection
type 2 header 0x8C + Slot68 records
type 3 positions float3
type 4 packed normals
type 5 packed UV0
type 6 index buffer u16
type 7 triangle descriptors
type 8 animation keys
type 9 service stream
type 10 strings and node names
type 13 Batch20 records
type 15 auxiliary stream
type 17 auxiliary data
type 18 rare stream
type 19 animation frame map
type 20 rare auxiliary table
```
Reader ищет entries по type, но сохраняет исходный порядок для roundtrip.
## Node and slot selection
Type 1 обычно состоит из records по 38 bytes:
```c
struct Node38 {
uint16_t hdr0;
uint16_t parent_or_link;
uint16_t anim_map_start;
uint16_t fallback_key;
uint16_t slot_index[15];
};
```
`slot_index[lod * 5 + group]` выбирает geometry slot. `0xFFFF` означает
отсутствие геометрии для комбинации LOD/group.
## Slot and batch
Type 2 содержит header `0x8C`, затем `Slot68`:
```c
struct Slot68 {
uint16_t tri_start;
uint16_t tri_count;
uint16_t batch_start;
uint16_t batch_count;
float aabb_min[3];
float aabb_max[3];
float sphere_center[3];
float sphere_radius;
uint32_t opaque[5];
};
```
Type 13 задаёт draw ranges:
```c
#pragma pack(push, 1)
struct Batch20 {
uint16_t batch_flags;
uint16_t material_index;
uint16_t opaque4;
uint16_t opaque6;
uint16_t index_count;
uint32_t index_start;
uint16_t opaque14;
uint32_t base_vertex;
};
#pragma pack(pop)
```
Index check выполняется как `base_vertex + index < vertex_count` для всего
используемого slice.
+61
View File
@@ -0,0 +1,61 @@
# NRes
`NRes` -- основной контейнер ресурсов Iron3D. Он используется как внешний
архив и как внутренний контейнер модели `*.msh`.
```text
[Header: 16 bytes]
[Data region: payload with alignment]
[Directory: entry_count * 64 bytes]
```
## Header
```c
struct NResHeader16 {
char magic[4]; // "NRes"
uint32_t version; // 0x00000100
int32_t entry_count; // >= 0
uint32_t total_size; // equals file size
};
```
`directory_offset = total_size - entry_count * 64`. Reader проверяет отсутствие
переполнений, `directory_offset >= 16` и точное окончание каталога на
`total_size`.
## Entry
```c
#pragma pack(push, 1)
struct NResEntry64 {
uint32_t type_id;
uint32_t attr1;
uint32_t attr2;
uint32_t size;
uint32_t attr3;
char name[36];
uint32_t data_offset;
uint32_t sort_index;
};
#pragma pack(pop)
```
Имя содержит bounded C-string до 35 полезных bytes. `sort_index` задаёт
отображение из sorted position в original entry index. В строгом режиме все
`sort_index` образуют перестановку `0..N-1`.
## Data region
Payload каждой записи лежит после header и до начала каталога. Игровые архивы
выравнивают следующий payload до 8 bytes нулями, но reader не должен требовать
плотного покрытия data region.
Различаются:
- active payload -- диапазон, на который указывает entry;
- gap/padding -- bytes между активными диапазонами;
- unindexed preserved region -- произвольные bytes, не принадлежащие entry.
Lossless editor сохраняет все три категории. Compact writer может исключить
unindexed regions только при явной операции repack.
+56
View File
@@ -0,0 +1,56 @@
# Render frame
Кадр является последней стадией цикла, а не самостоятельной функцией renderer-а.
До draw calls уже накоплен input, рассчитан tick, применены отложенные операции,
выбрана камера и обновлён 3D sound listener.
## Frame skeleton
```text
system messages and input
-> simulation calculation
-> deferred object operations
-> animation and transforms
-> camera and sound listener
-> visibility and render queues
-> materials and draw passes
-> renderer completion
-> end-of-render callbacks and UI
```
В `World3D::stdRenderGame` доказан крупный порядок: camera передаётся Terrain,
настраиваются viewport/matrices, вызываются renderer boundary slots,
устанавливается `in_render`, выполняется traversal мира, закрывается world/shade
pass, вызывается renderer completion, снимается `in_render`, рассылается
end-of-render.
## Draw item
Подготовленный draw item содержит:
- node world matrix;
- batch flags and index range;
- WEAR material handle;
- MAT0 active phase and coefficients;
- texture handle;
- optional lightmap handle;
- render phase and sorting key;
- legacy pipeline state.
Подготовленный item должен ссылаться на immutable данные кадра. Изменение phase
или texture cache посреди прохода не должно менять уже собранную очередь.
## Parity risks
- x87 precision and rounding;
- scalar/SIMD `g_FastProc` differences;
- object, batch and transparent primitive order;
- depth, cull, alpha test and blend transitions;
- mip-skip, palette and Page coordinates;
- material fallback and phase selection;
- RNG sequence for FX and atmosphere;
- device capability fallback;
- simulation time quantization.
Для отладки нужен deterministic frame capture: camera state, visible object IDs,
draw-item list, pipeline keys, matrices и hashes промежуточных buffers.
+69
View File
@@ -0,0 +1,69 @@
# RsLi
`RsLi` -- библиотечный архив Iron3D с каталогом в начале файла и payloads после
него.
```text
[Header: 32 bytes]
[Entry table: entry_count * 32 bytes]
[Payloads]
[optional trailer]
```
## Header fields
```text
+0x00 char[2] "NL"
+0x02 u8 reserved
+0x03 u8 version = 1
+0x04 i16 entry_count
+0x0E u16 presorted_flag = 0xABBA
+0x14 u32 xor_seed
```
Остальные bytes сохраняются без нормализации.
## Entry
```c
struct RsLiEntry32 {
char name[12];
uint8_t service[4];
int16_t flags;
int16_t sort_to_original;
uint32_t unpacked_size;
uint32_t data_offset_raw;
uint32_t packed_size;
};
```
Имя обычно хранится в uppercase ASCII. `sort_to_original` связывает sorted
position с исходной записью.
## Table transform
Entry table проходит обратимое потоковое XOR-преобразование. Начальное
состояние берётся из младших 16 bits `xor_seed` и продолжается через всю
таблицу, не сбрасываясь на границе записи.
## Storage methods
```text
0x000 raw block
0x020 byte transform only
0x040 LZSS
0x060 transform + LZSS
0x080 adaptive Huffman + LZSS
0x0A0 transform + adaptive Huffman + LZSS
0x100 raw Deflate
```
После любого пути должно получиться ровно `unpacked_size` bytes. Методы
`0x080` и `0x0A0` подтверждены decoder-кодом, но не живыми payload демоверсии
или обеих частей.
## Compatibility quirk
`sprites.lib::INTERF8.TEX` объявляет Deflate range на один byte дальше EOF.
Совместимый reader допускает `packed_size - 1` только для этого именованного
случая. Строгий режим сообщает `deflate_eof_plus_one`.

Some files were not shown because too many files have changed in this diff Show More