Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Architecture

Technical overview of HPM — system design, dependency resolution, cleanup, storage, and Python integration. Intended for contributors and integrators.

Table of contents

  1. System overview
  2. Crate layout
  3. Core types
  4. Dependency resolution
  5. Install flow
  6. Project-aware cleanup
  7. Storage architecture
  8. Python integration
  9. Security and performance
  10. Extension points

System overview

HPM is a modular monolith. A single binary (hpm-cli) drives a handful of focused library crates. Responsibilities are split along stable seams: manifest parsing, configuration, resolution, storage, Python, and errors all live in their own crates so they can be reused (e.g., a desktop app can depend on the library crates and skip the CLI entirely).

┌────────────────────────────────────────────────────────────────────────┐
│ CLI (hpm-cli)                                                          │
│   • subcommands: init, add, remove, install, update, list, search,     │
│     run, check, build, pack, audit, clean, registry, global,          │
│     completions                                                        │
│   • output formats: human, json, json-lines, json-compact              │
│   • shell completions: bash, zsh, fish, powershell, elvish             │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │
┌───────────────────────────────────▼────────────────────────────────────┐
│ Core orchestration (hpm-core)                                          │
│   • StorageManager: global package install/remove/list                 │
│   • ProjectDiscovery: scan configured paths for active projects        │
│   • LockFile / LockedDependency / PackageSource                        │
│   • Registry trait + ApiRegistry, GitRegistry                          │
│   • ArchiveFetcher, packer                                             │
└─────────┬──────────────────┬──────────────────┬─────────────────────────┘
          │                  │                  │
┌─────────▼────────┐   ┌───────────────┐   ┌───────────────┐
│ hpm-package      │   │ hpm-assets    │   │ hpm-config    │
│   PackageManifest│   │   Asset       │   │   Config      │
│   DependencySpec │   │   AssetKind   │   │   Storage/    │
│   StageConfig    │   └───────────────┘   │   Projects/   │
│   Platform       │                       │   Signing     │
│   HoudiniPackage │                       └───────────────┘
└──────────────────┘

Python tooling — bundled uv, content-addressable venvs, the Houdini→Python ABI mapping, and per-script venv resolution — lives in hpm_core::python. It was a separate hpm-python crate through 0.16 but folded into hpm-core since it had no consumers other than hpm-core and hpm-cli.

hpm-package and hpm-assets are the workspace leaves — neither depends on another HPM crate. hpm-config depends on hpm-package (it reuses the manifest’s registry and dependency types), so it is not a leaf despite sitting alongside them in the diagram above.

Each crate defines its own error type (e.g. StorageError, ConfigError) via thiserror. Errors surface to the user through CliError in hpm-cli, which converts them to exit codes and help hints.

Key non-functional properties:

  • Language: Rust, edition 2024, MSRV 1.85.
  • Async runtime: Tokio, multi-thread.
  • Configuration: TOML throughout (manifests, lock file, global config).
  • Testing: unit, integration, and property-based (proptest) tests.

Crate layout

CrateResponsibility
hpm-cliCommand-line frontend (clap). Turns command-line invocations into calls on the library crates.
hpm-coreStorage, project discovery, lock file, registry trait + two implementations, archive fetching/packing, Python tooling (python submodule: bundled uv, venvs, cleanup).
hpm-packagehpm.toml parsing and validation, dependency/Python dependency types, Houdini package.json generation, platform enum.
hpm-assetsOperator asset-index model (Asset, AssetKind) emitted by hpm pack from [[operators]] declarations.
hpm-configGlobal and project configuration loading and merging.

Dependencies flow downward: hpm-cli depends on everything else, hpm-core depends on package, assets, and config, and so on. No crate depends upward. Domain errors (StorageError, ConfigError, …) live in the crate that produces them.

Core types

Selected public types. See cargo doc --workspace --no-deps --open for the full list.

hpm-package

#![allow(unused)]
fn main() {
pub struct PackageManifest {
    pub package: PackageInfo,
    pub compat: CompatConfig,
    pub stage: StageConfig,
    pub registries: Vec<RegistryConfig>,
    pub dependencies: IndexMap<String, DependencySpec>,
    pub python_dependencies: IndexMap<String, PythonDependencySpec>,
    pub runtime: IndexMap<String, ManifestEnvEntry>,
    pub scripts: PackageScripts,
    pub operators: Vec<OperatorDecl>,
}
// All collections and section structs use `#[serde(default,
// skip_serializing_if = ...)]` so an absent TOML section round-trips to
// the same in-memory representation as an empty one. `Option` was
// removed in 0.17 — see CHANGELOG.

pub struct PackageScripts {
    pub commands: IndexMap<String, ScriptEntry>,  // flat entries under [scripts]
}

// Untagged: serialises as either a bare string ("cargo build") or a table
// form { cmd, python?, requirements?, label?, description?, package-env? }.
// The table form opts the script into
// a uv-managed venv resolved on demand by `hpm run`, and supports
// conditional cmd values via the same `when` grammar `[runtime]` uses.
pub enum ScriptEntry {
    Plain(String),
    WithEnv(ScriptEnv),
}

pub struct ScriptEnv {
    pub cmd: EnvValue,             // flat string or [{ when = { os }, set }, ...]
    pub python: Option<String>,        // e.g. "3.11"
    pub requirements: Vec<String>,     // e.g. ["PySide6>=6.6"]
    pub label: Option<String>,         // display name for UIs; hpm never acts on it
    pub description: Option<String>,   // one-line description for tooltips/help
    pub package_env: bool,             // run inside the package's full resolved env
}

pub struct PackageInfo {
    pub path: PackagePath,         // "creator/slug", validated kebab-case
    pub name: String,              // freeform display name
    pub version: String,           // semver
    pub description: Option<String>,
    pub authors: Vec<String>,      // empty vec encodes "no authors declared"
    pub license: Option<String>,
    pub readme: Option<String>,
    pub homepage: Option<String>,
    pub repository: Option<String>,
    pub documentation: Option<String>,
    pub keywords: Vec<String>,
    pub categories: Vec<String>,
}

pub enum DependencySpec {
    // The bare-string shorthand `pkg = "1.0.0"` (de)serializes as
    // Registry { registry: None, optional: false }.
    Registry { version: String, registry: Option<String>, optional: bool },
    Url { url: String, version: String, optional: bool },
    Path { path: String, optional: bool, link: bool },       // link=true → symlink/junction install
}

pub enum Platform {
    LinuxX86_64,       // "linux-x86_64"
    LinuxAarch64,      // "linux-aarch64"
    MacosX86_64,       // "macos-x86_64"
    MacosAarch64,      // "macos-aarch64"
    WindowsX86_64,     // "windows-x86_64"
    WindowsAarch64,    // "windows-aarch64"
    Universal,         // "universal" — OS-agnostic (pure-Python / data)
}
}

hpm-core

#![allow(unused)]
fn main() {
pub struct LockFile {
    pub package: LockPackageInfo,
    pub dependencies: BTreeMap<String, LockedDependency>,
    pub python_dependencies: BTreeMap<String, LockedPythonDependency>,
    pub metadata: Option<LockMetadata>,
}

pub struct LockedDependency {
    pub version: String,
    pub checksum: Option<String>,    // "sha256:<hex>"
    pub source: LockedSource,        // Url { url, version } | Path { path }
}

#[async_trait]
pub trait Registry: Send + Sync {
    async fn search(&self, query: &str) -> Result<SearchResults, RegistryError>;
    async fn get_versions(&self, name: &str) -> Result<Vec<RegistryEntry>, RegistryError>;
    async fn get_version(&self, name: &str, version: &str) -> Result<RegistryEntry, RegistryError>;
    async fn refresh(&self) -> Result<(), RegistryError>;
    fn name(&self) -> &str;
}
}

hpm-core::python

#![allow(unused)]
fn main() {
pub struct PythonVersion {
    pub major: u8,
    pub minor: u8,
    pub patch: Option<u8>,
}

pub struct ResolvedDependencySet {
    pub python_version: PythonVersion,
    pub packages: BTreeMap<String, String>,   // name → exact version
}

pub struct VenvManager { /* … */ }

impl VenvManager {
    pub async fn ensure_virtual_environment(
        &self,
        resolved: &ResolvedDependencySet
    ) -> Result<PathBuf, PythonError>;
}
}

hpm-config

#![allow(unused)]
fn main() {
pub struct Config {
    pub install: InstallConfig,
    pub storage: StorageConfig,
    pub projects: ProjectsConfig,
    pub registries: Vec<RegistrySourceConfig>,
    pub signing: SigningConfig,
}

pub struct StorageConfig {
    pub home_dir: PathBuf,            // default: $HOME/.hpm
    pub cache_dir: PathBuf,
    pub packages_dir: PathBuf,
    pub registry_cache_dir: PathBuf,
}
}

Configuration is layered: defaults → ~/.hpm/config.toml<cwd>/.hpm/config.toml. Files are parsed as presence-aware overlays (ConfigOverlay), so each layer overrides exactly the values it sets — a file that omits a section leaves the lower layers untouched.

Dependency resolution

HPM currently does naive per-package version selection: for each declared registry dependency, query the registry, filter non-yanked versions, and pick the highest that matches the spec’s VersionReq. There is no transitive constraint solving — every dependency is treated as a direct dependency, and registry packages don’t declare their own deps in a way the install path consumes. A proper constraint solver (PubGrub-style or otherwise) will be reintroduced if/when transitive resolution becomes a real requirement.

Version requirement grammar

HPM manifest values accept the usual semver constraint operators:

FormMeaning
=1.2.3, 1.2.3Exact.
^1.2.3Compatible with the given major (>=1.2.3, <2.0.0).
~1.2.3Patch updates allowed (>=1.2.3, <1.3.0).
>=1.0.0, <2.0.0Explicit range.

Install flow

The install command itself is a thin shell — it loads the manifest + lockfile, builds a ProjectManager, and calls sync_dependencies(). The desktop client uses the same ProjectManager entry point, so both clients run the same flow.

 ┌──────────────────────────────────────────────────────────────────────┐
 │ hpm install (CLI)            ── ProjectManager::sync_dependencies ── │
 ├──────────────────────────────────────────────────────────────────────┤
 │  1. Load hpm.toml                                                    │
 │  2. If hpm.lock exists:                                              │
 │       verify cached packages against stored checksums                │
 │       warn if metadata.generated_at > 90 days                        │
 │  3. Fetch + install in parallel (one task per dep, JoinSet):         │
 │       Registry        → query registry, exact-version lookup         │
 │                       → ArchiveFetcher → ~/.hpm/fetch/               │
 │                       → install_into_cas → ~/.hpm/packages/<slug>@<v>/ │
 │       Url             → ArchiveFetcher (no registry query)           │
 │                       → install_into_cas                             │
 │       Path            → install_as_dev_copy (or install_as_dev_link  │
 │                         if { link = true })                          │
 │                       → ~/.hpm/packages/_dev/<slug>@<v>/             │
 │                         (real dir for copy, symlink/junction for     │
 │                         link mode — both honor _dev/ CAS isolation)  │
 │     Already-in-CAS deps short-circuit (avoids the install_into_cas   │
 │     remove-and-recopy that breaks on Windows when Houdini is open).  │
 │  4. Merge [python_dependencies] from root + every dep manifest       │
 │     Python ABI = root manifest's [compat].houdini lower bound        │
 │  5. Ensure managed CPython installed under ~/.hpm/uv-python/         │
 │     (uv python install <ver>) — auto-downloads on clean machines     │
 │  6. Resolve with bundled uv, hash the resolved set, pick or          │
 │     rebuild ~/.hpm/venvs/<hash>/                                     │
 │  7. uv pip install --python <venv>/bin/python                        │
 │  8. Write <project>/.hpm/packages/{creator}.{slug}.json per dep      │
 │  9. Sweep stale <project>/.hpm/packages/ entries                     │
 │ 10. CLI step: build new lockfile from sync's InstallOutcomes         │
 │     (backfilled from the prior lockfile for short-circuited entries) │
 │     and write hpm.lock                                               │
 └──────────────────────────────────────────────────────────────────────┘

Note: registry resolution currently uses each dep spec’s version string as an exact lookup against the registry. Range specs like ^1.0.0 in [dependencies] won’t resolve through hpm install today — hpm update is the path that re-resolves ranges (querying all versions, filtering by VersionReq, picking the highest non-yanked) and rewrites the spec to the resolved exact version.

--frozen-lockfile aborts if loading the existing lockfile fails, if any cached checksum mismatches, or if the freshly-resolved set differs from the prior lockfile.

Project-aware cleanup

hpm clean removes packages and venvs no active project depends on. The identity of “active projects” comes from the [projects] config section.

  I = installed packages in ~/.hpm/packages/
  P = active projects discovered via ProjectDiscovery
  D(p) = direct deps declared by project p
  T(I) = transitive closure over the dependency graph

  orphans = I \ T( ⋃ D(p) )

Implementation outline:

#![allow(unused)]
fn main() {
// crates/hpm-core/src/graph.rs
impl DependencyGraph {
    /// BFS from each root, returning every `PackageId` reachable along
    /// dependency edges. The roots themselves are included.
    pub fn mark_reachable_from_roots(&self, roots: &[PackageId]) -> HashSet<PackageId> {
        let mut reachable = HashSet::new();
        for root in roots {
            let Some(&start) = self.index_of.get(root) else {
                continue;
            };
            let mut bfs = Bfs::new(&self.graph, start);
            while let Some(idx) = bfs.next(&self.graph) {
                reachable.insert(self.graph[idx].id.clone());
            }
        }
        reachable
    }
}
}

DependencyGraph wraps a petgraph graph; its fields are private and it exposes no orphans() of its own. Taking the set difference against what is actually installed on disk is StorageManager::find_orphaned_packages (crates/hpm-core/src/storage/cleanup.rs), which calls mark_reachable_from_roots and subtracts the result from the installed set.

Project discovery scans explicit_paths plus recursive walks of search_roots, respecting max_search_depth and skipping directory names that match ignore_patterns. Only directories containing an hpm.toml are considered projects.

Python venv cleanup uses the same principle against the metadata.json files in each venv, which list the HPM packages using that venv.

Storage architecture

HPM stores everything under ~/.hpm/ on every supported platform. The layout is content-addressable where it helps:

~/.hpm/
├── config.toml
├── packages/                       # canonical CAS — written by StorageManager
│   ├── slug@1.0.0/                 # registry / URL installs
│   │   ├── hpm.toml
│   │   ├── (package sources)
│   │   └── … (Houdini convention subdirs)
│   └── _dev/                       # path-installed (dev) packages
│       └── slug@1.0.0/             # never substituted for a registry hit
│           └── …                   # at the same (slug, version)
├── fetch/                          # ArchiveFetcher staging — extracted
│   └── creator-slug-1.0.0/         # archives live here briefly before
│                                   # install_into_cas copies into packages/
├── venvs/
│   └── <12-char hash>/             # hash of resolved set + Python version
│       ├── pyvenv.cfg
│       ├── bin/                    # or Scripts/ on Windows
│       ├── lib/pythonX.Y/site-packages/
│       └── metadata.json
├── cache/                          # download archive cache
├── registry/                       # one subdir per registry
│   └── <registry name>/
├── tools/                          # bundled uv binary
├── uv-cache/                       # isolated uv cache
├── uv-config/
├── uv-python/                      # managed CPython installs (UV_PYTHON_INSTALL_DIR)
└── fetch/                          # staging for downloaded archives

hpm install routes URL/registry deps through a two-step flow: ArchiveFetcher downloads + extracts into ~/.hpm/fetch/, then StorageManager::install_into_cas copies into the canonical CAS at ~/.hpm/packages/<slug>@<version>/. Path deps skip the fetcher entirely and go straight to ~/.hpm/packages/_dev/<slug>@<version>/ via install_as_dev_copy.

Per-project:

<project>/
├── hpm.toml
├── hpm.lock
└── .hpm/
    ├── config.toml                 # optional
    └── packages/
        ├── utility-nodes.json      # generated Houdini manifests, one per dep
        └── material-library.json   # absolute paths into the global CAS

Global packages are shared across projects; each project holds only the per-dependency Houdini manifest and a lockfile. This keeps disk usage sane across a studio’s worth of projects. The Houdini JSON manifests carry absolute CAS paths, so the project tree contains no symlinks pointing into the global storage.

Each per-package manifest is named <creator>.<slug>.json. The creator segment is part of the filename because the slug alone is not unique: two creators may publish the same slug, and keying the file on the slug let the second install silently overwrite the first, so Houdini loaded only one of them. . is the separator rather than - because both segments may themselves contain - (a-b/c and a/b-c would collide), while . is not a legal segment character.

sync_dependencies sweeps stale per-package manifests at the end of every install run: any .json in <project>/.hpm/packages/ whose stem is no longer in the resolved dependency set is removed. Without this, a manifest written by a previous run (e.g. for a path dependency that has since been removed) would keep loading the package on Houdini launch even though hpm.toml no longer asks for it. Manifests written by an older hpm under the bare <slug>.json name are not in the current scheme, so the same sweep clears them on the next install — otherwise the package would load twice, once per filename. Note this makes <project>/.hpm/packages/ an hpm-owned directory: unrecognized .json files placed there by hand do not survive a sync.

Global installs

hpm global writes the same Houdini manifest into Houdini’s per-user preferences directory (hpm_core::houdini_prefs resolves it per platform, honouring HOUDINI_USER_PREF_DIR and its __HVER__ placeholder) instead of into a project. It reuses the whole install path — the same RegistrySet, the same CAS, the same shared venvs, and the same manifest builder (build_houdini_package, a free function precisely so both callers produce byte-identical output).

The one structural difference is directory ownership, and it inverts the sweep described above. <project>/.hpm/packages/ belongs to hpm, so the project installer may delete anything it does not recognise there. Houdini’s user packages/ directory belongs to the user: it holds files written by SideFX, by other tools, and by hand. Applying the sweep there would delete them. So global installs are tracked in a ledger at ~/.hpm/global/houdini-<X.Y>.json, and every operation is driven off it — list reads it, remove deletes exactly the filename it records, and nothing ever scans the directory to decide what to delete.

The ledger is also a GC root. find_orphaned_packages marks reachability from discovered projects, and a globally installed package belongs to no project; without its ledger entry it is unreferenced by construction, so cleanup would delete the store directory while the manifest in Houdini’s preferences kept pointing at it. An unreadable ledger aborts cleanup rather than being treated as empty — “no global installs” and “cannot tell” must not look the same to a collector.

Compatibility is checked before anything is written. The emitted enable expression is evaluated by Houdini at startup, so a package whose [compat].houdini excludes the target version would install successfully and then load nothing, silently. check_compatible turns that into an install- time error.

Path dependencies install into ~/.hpm/packages/_dev/<slug>@<version>/ rather than the registry CAS at ~/.hpm/packages/<slug>@<version>/. The _dev subtree is invisible to list_installed, so a dev install of foo@1.0.0 cannot be served as the cached install for a registry resolution at the same coordinate from a different project.

_dev/ entries are garbage-collected by a parallel cleanup pass driven directly off project path-dependencies (since the CAS-orphan logic deliberately can’t see them). For each discovered project, find_orphaned_dev_installs parses the manifest, resolves every { path = "..." } dep’s source manifest to its (slug, version), and unions those into the “needed” set. Anything in _dev/ whose dir-name encoding falls outside that set is orphan. A project with an unreadable path-dep source logs a warning and skips that dep — a broken project doesn’t bypass cleanup of other dev installs; re-running hpm install re-creates whatever the project still needs. Removal goes through the same remove_install_entry primitive used by clear_existing_install and remove_package, so link installs are unlinked without traversal.

Path deps come in two install styles, selected by the link field on the manifest’s { path = "...", link = ? } spec:

  • Copy (default, link = false): install_as_dev_copy snapshot-copies the workspace into a content-addressed _dev/<slug>@<version>/<source-hash>/ directory (see below). Subsequent working-tree edits don’t reach the install until the next hpm install.
  • Link (link = true): install_as_dev_link creates a symlink (Unix) or NTFS junction (Windows) at _dev/<slug>@<version>/ pointing at the canonicalized workspace. Edits are live; Houdini’s HPATH resolution follows the link transparently. Junctions on Windows side-step the Developer Mode / admin requirement that NTFS directory symlinks carry, so the workflow is viable on a stock Houdini workstation.

Native-binary packages are the exception: when a link = true path dep declares concrete [compat].platforms (anything other than universal), install_inner downgrades it to a copy — see CompatConfig::declares_native_platforms. A junction/symlink would make the workspace’s build output the very DSO/DLL a running Houdini has memory-mapped, so an in-place rebuild fails (Windows LNK1104 / ERROR_SHARING_VIOLATION), and link-mode buys nothing for native code anyway — a mapped DSO can’t hot-reload into a live session; it needs a relaunch, which re-copies and re-runs prepack. Pure-data / pure-Python / universal-only link installs are unaffected.

Dev copies are content-addressed so a re-sync never clears a directory a running Houdini has mapped. source_hash fingerprints the workspace (relative path + length + mtime of every file, stat-only) and that hash names the install directory: the copy lands at _dev/<slug>@<version>/<source-hash>/, and the generated Houdini manifest points hpath there. install_inner stages the copy into a hidden .stage-<pid>-<n> directory and commits it with an atomic, race-tolerant rename (a concurrent install that loses the race drops its stage, since the content is identical by construction). If the target hash directory is already present and complete, the existing install is reused untouched — the copy-mode analogue of the “already installed” short-circuits the registry/URL specs have in install_one_dep.

The payoff is that a rebuild produces a new hash directory rather than overwriting the old one: a concurrently-running Houdini keeps mapping the directory it was launched from, so install_inner never remove_dir_alls a live copy. Before content addressing, an in-place clear-and-recopy of a package a second session still had mapped failed on Windows with os error 5 / StorageError::PackageInUse; that failure is now structurally impossible on all platforms. At install time only a stale link left by a prior link = true install is cleared (clear_container_link, link-safe); the directory of live hashes is left in place.

Superseded hash directories accumulate across a dev-iteration loop and are reclaimed by hpm clean: cleanup_unused_dev_installs removes whole orphan containers (tolerant of in-use) and, for still-referenced installs, prunes every hash directory except the one matching the current source (prune_stale_dev_hashes). Reclamation is best-effort and carries the same “run when Houdini sessions are closed” expectation as CAS package cleanup — a copy still mapped by a live process is skipped (on Windows the OS lock fails the removal) rather than force-removed.

Both install replacement (clear_existing_install) and orphan cleanup (remove_package) are symlink-safe: each checks symlink_metadata (plus junction::exists on Windows) before deciding between remove_dir_all (real dir) and remove_file / junction::delete (link entry), via a shared remove_install_entry primitive. Without this, a remove_dir_all on a Windows junction would recurse into and delete the user’s workspace on the next sync or orphan sweep.

Both styles set InstalledPackage::is_dev = true. That flag flows through to create_houdini_package_with_python, which forwards it to ManifestEnvEntry::lower(.., is_dev). The install_source axis on each conditional [runtime] variant is filtered there: variants gated to "dev" only survive for path-installed packages, variants gated to "registry" only survive for registry/URL installs, and variants with no axis fire in both contexts. A registry-fetched install whose every matching variant is install_source = "dev" produces no entry for that key in the generated package.json, so dev-only paths never leak to a published consumer.

Emission targets Houdini’s actual env semantics, as verified with the hconfig conformance harness (hpm-core/src/houdini_conformance_tests.rs) and modeled executably in hpm-core/src/houdini_env_model.rs: prepend / append values are single-element JSON lists (a flat-string first definition makes a custom variable non-mergeable — every later entry overwrites it, whatever its method), set is emitted as a bare flat string (Houdini has no set method; a list-form replace appends onto a path-registered variable Houdini seeded flat-first, e.g. OCIO, so only a flat string overwrites it cleanly — a genuinely conditional set still falls back to replace since it can’t be flat), and conditional branches are compiled mutually exclusive because Houdini applies every matching conditional-array element, not the first match. Project-level [runtime] entries are written once to ~hpm-project-overrides.json in the project packages dir; Houdini processes package files in byte-wise ascending filename order and ~ sorts after every slug character, so that file always applies last — an override lands exactly once, after (or, for set, replacing) every package contribution.

Python integration

The Python layer runs on four ideas, in descending order of importance:

  1. Content-addressable venvs. Hash the resolved dependency set, use it as the venv directory name. Same hash → same venv → shared across packages.
  2. Bundled uv. A copy of uv ships with HPM and lives at ~/.hpm/tools/uv. Its cache (~/.hpm/uv-cache/), config (~/.hpm/uv-config/), and managed CPython installs (~/.hpm/uv-python/, pinned via UV_PYTHON_INSTALL_DIR) are isolated from any system uv so HPM never perturbs your other Python work.
  3. Self-bootstrapping Python. uv pip compile and uv venv need an interpreter. Before invoking either, HPM runs uv python install <ver> to ensure a managed CPython matching the project’s Houdini ABI exists. This is what makes a clean Windows install (no system Python anywhere) Just Work — without it pip compile errors with No interpreter found in virtual environments, managed installations, search path, or registry. The ensure step is process-cached, so it costs one fast filesystem probe per resolution.
  4. Houdini manifest generation. For each HPM package that declares Python dependencies, HPM writes <project>/.hpm/packages/{creator}.{slug}.json with PYTHONPATH prepended at the shared venv’s site-packages.

Hash function

#![allow(unused)]
fn main() {
// crates/hpm-core/src/python/types.rs
impl ResolvedDependencySet {
    pub fn hash(&self) -> String {
        let mut hasher = Sha256::new();
        hasher.update(format!("python:{}", self.python_version));
        for (name, version) in &self.packages {
            hasher.update(format!("{}:{}", name, version));
        }
        hasher
            .finalize()
            .iter()
            .map(|b| format!("{:02x}", b))
            .collect::<String>()[..16]
            .to_string()
    }
}
}

packages is a BTreeMap<String, String>, so it iterates in sorted order without an explicit sort. Only names and versions are hashed — extras are not part of the resolved set.

Houdini→Python mapping

HoudiniPython
20.5+3.11
21.x3.11
22.x3.13

The mapping is sourced from the lower bound of the root manifest’s [compat].houdini — the project’s Houdini build is what determines the embedded CPython ABI that wheels in the venv must match. A dependency package’s own [compat].houdini describes a compatibility floor (the oldest Houdini it runs on) and is not consulted for ABI selection: a [compat].houdini = ">=21.0" package consumed by a Houdini-22 project still gets a 3.13 venv.

Unsupported versions return an error. No silent fallback — an ABI-mismatched venv would break C-extension imports at Houdini launch instead of surfacing the mapping gap at install time. Houdini 19.x (Python 3.7) and 20.0–20.4 (Python 3.9) are unsupported: their Python interpreters are past upstream EOL.

Generated Houdini manifest

{
  "hpath": ["/Users/me/.hpm/packages/studio/my-tool@1.0.0"],
  "env": [
    {
      "PYTHONPATH": {
        "method": "prepend",
        "value": "/Users/me/.hpm/venvs/a1b2c3d4e5f6/lib/python3.11/site-packages"
      }
    }
  ],
  "enable": "houdini_version >= '20.5'"
}

method: "prepend" delegates path-separator handling to Houdini so the same manifest works on Windows (;) and Unix (:) without embedding an OS-specific joiner. The generator is hpm_core::project::houdini_emit::build_houdini_package, a free function shared by the project installer and hpm global so both emit identical manifests.

Security and performance

Security

  • Transport: TLS through rustls (pure Rust, no OpenSSL). Certificate verification against the platform trust store.
  • Integrity: SHA-256 over every installed archive, recorded in hpm.lock, verified before every install.
  • Signing: hpm pack --key signs archives with Ed25519 over PKCS#8 PEM private keys. keyId = first 8 bytes of public key, hex-encoded. See Security for the wire format.
  • Isolation: bundled uv runs out-of-process with its own cache and config, never touching system state.

Performance

  • Concurrent downloads: registry-fronted installs run in parallel, bounded by [install].parallel_downloads (default 8).
  • Content-addressable dedup: both HPM packages (keyed by slug@version in the global CAS) and Python venvs (keyed by resolved-set hash) are deduplicated globally.
  • Cache hits: uv’s cache eliminates wheel re-downloads across projects; HPM’s venv cache eliminates install work when a matching set is found.
  • Atomic filesystem ops: packages are extracted to a temp directory and renamed into place to avoid partial states.

Extension points

Programmatic configuration

The library crates can be used without the CLI. Embedders (desktop apps, pipeline tools) build a Config in-memory by starting from Config::default() and setting the fields they need:

#![allow(unused)]
fn main() {
use hpm_config::{Config, RegistrySourceConfig, RegistryType};

let mut config = Config::default();
config.add_registry(RegistrySourceConfig {
    name: "tumbletrove".into(),
    url: "https://api.tumbletrove.com/v1/registry".into(),
    registry_type: RegistryType::Api,
});
config.install.path = "packages/hpm".into();
}

Registry trait

Any type that implements the Registry async trait can be plugged into a RegistrySet and used for resolution. The built-in ApiRegistry and GitRegistry are reference implementations.

For visibility-gated API registries (e.g. private packages an authenticated user is entitled to see), embedders can pass a bearer token through RegistrySet::from_configs_with_auth — the token is attached as Authorization: Bearer <token> on every API-registry request and marked sensitive so reqwest won’t log it. The token is baked into the HTTP client at construction, so callers tracking a refreshing token rebuild the RegistrySet per operation rather than mutating one in place. Git registries currently ignore the token.

ProjectManager mirrors that contract via ProjectManager::new_with_auth(.., Option<String>). The token is stashed on the manager and forwarded to every RegistrySet it builds internally — sync_dependencies for registry-form deps and add_dependency’s registry-resolved path both go through from_configs_with_auth with the stashed token. ProjectManager::new delegates with None, so anonymous use is unchanged. As with the RegistrySet variant, refreshing tokens are handled by rebuilding the ProjectManager per operation.

Plugin system

Not implemented. The CLI surface is currently fixed at compile time. A dynamic plugin system is on the roadmap but not prioritized — most studios are better served by wrapping HPM in a higher-level pipeline tool.