Ericson2314/nix-roadmap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'pdt'

Browse Source
This commit is contained in:
John Ericson 2025-04-17 16:52:54 -04:00
commit c5b565d5ac
1 changed files with 8 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
README.md
View File

@ -60,7 +60,7 @@ I don't want content-addressing derivations to be responsible for any of those.
Drilling deeper, what does "ensuring the binary cache is sound" entail? Drilling deeper, what does "ensuring the binary cache is sound" entail?
I think the essential issue is [Nix#11896]. I think the essential issue is [Nix#11896].
"deep realisations" --- build trace key-value pairs where the key includes derivations that depend on other derivations' outputs --- are fundamentally ambiguous. "deep realisations" --- build trace key-value pairs where the key includes derivations that depend on other derivations' outputs --- are fundamentally ambiguous.
This ambiguous makes them hard to verify/challenge, and hard to know when they conflict --- two deep realisations may implicitly make incompatible assumptions about the outputs of those dependency derivations. This ambiguity makes them hard to verify/challenge, and hard to know when they conflict --- two deep realisations may implicitly make incompatible assumptions about the outputs of those dependency derivations.
We currently have a notion of "dependent realisations" that seeks to address this issue, but I do not think this mechanism is sound, and it is certainly not consistently implemented. We currently have a notion of "dependent realisations" that seeks to address this issue, but I do not think this mechanism is sound, and it is certainly not consistently implemented.
The simplest thing to do is....just rip out deep realisations. The simplest thing to do is....just rip out deep realisations.
@ -82,10 +82,10 @@ We can always delete *all* realisations locally
(there are no hard references between shallow realisations -- no "closure property"), (there are no hard references between shallow realisations -- no "closure property"),
so that sledgehammer can always be presented as a fail-safe last resort to unbreak anyone's machine that ran out of disk space. so that sledgehammer can always be presented as a fail-safe last resort to unbreak anyone's machine that ran out of disk space.
Again, the current way we GC realisations (leveraging those "dependent realisations") is not necessarily a good or the only way to do things Again, the current way we GC realisations (leveraging those "dependent realisations") is not necessarily a good or the only way to do things
--- in fact, because the relationships between realisations are "soft" and not "hard", I very this as a situation where there are many possible "policies", and choosing between them is a matter of opinion. --- in fact, because the relationships between realisations are "soft" and not "hard", I see this as a situation where there are many possible "policies", and choosing between them is a matter of opinion.
Multiple policy/opinion territory is a clear place to cut scope for the first version. Multiple policy/opinion territory is a clear place to cut scope for the first version.
Downside two however I consider more series The second downside, however, I consider more serious
--- it would be really annoying to always download GCC whenever you just want some cached binary built with GCC. --- it would be really annoying to always download GCC whenever you just want some cached binary built with GCC.
Yes, you can GC that GCC right away, so there is no wasted disk space, but there is still the wasted time waiting for the download, and wasted network usage. Yes, you can GC that GCC right away, so there is no wasted disk space, but there is still the wasted time waiting for the download, and wasted network usage.
Downloading to then delete is not a solution, but just exposes how artificial and silly the status quo is. Downloading to then delete is not a solution, but just exposes how artificial and silly the status quo is.
@ -93,12 +93,12 @@ Downloading to then delete is not a solution, but just exposes how artificial an
[Nix#11928] is thus something I consider required to fix if we're going to get rid of deep realisations (as I propose). [Nix#11928] is thus something I consider required to fix if we're going to get rid of deep realisations (as I propose).
The good thing is that we can simply change the scheduling logic so it's no longer a problem. The good thing is that we can simply change the scheduling logic so it's no longer a problem.
The fix is conceptually simple enough: we can resolve derivations (normalize their inputs) without actually downloading those inputs. The fix is conceptually simple enough: we can resolve derivations (normalize their inputs) without actually downloading those inputs.
We just look up build trace key-value pairs and substitute within the derivation accordingly. We can just look up build trace key-value pairs and substitute within the derivation accordingly.
The less good news is that it is a bit harder than it sounds to implement, because the scheduling code was such a confusing mess. The less good news is that it is a bit harder than it sounds to implement, because the scheduling code is such a confusing mess.
#### Low level #### Low level
This in turn leans me to [Nix#12663]. This in turn leads me to [Nix#12663].
To make progress on the schedule code (and actually a bunch of other issues, which I'll hopefully get to), we need to untangle scheduling and building. To make progress on the schedule code (and actually a bunch of other issues, which I'll hopefully get to), we need to untangle scheduling and building.
Only then we'll we have a "clean workbench" upon which we can address reworking the scheduling logic for [Nix#11928] (and the other issues too). Only then we'll we have a "clean workbench" upon which we can address reworking the scheduling logic for [Nix#11928] (and the other issues too).
This might sound hard, but it actually isn't so bad --- it's just long overdue. This might sound hard, but it actually isn't so bad --- it's just long overdue.
@ -141,7 +141,7 @@ The linked issue contains a discussion of alternatives, I lean towards something
#### Rollout, Nixpkgs, RFC #### Rollout, Nixpkgs, RFC
Whereas the above is mostly "technical stuff I can just *do* without having to ask anyone for for permission", this part is squarely on community by-in. Whereas the above is mostly "technical stuff I can just *do* without having to ask anyone for for permission", this part is squarely on community buy-in.
I think what follows is a good process to follow, but, of course, no one knows for sure how the community will react until they do. I think what follows is a good process to follow, but, of course, no one knows for sure how the community will react until they do.
This is the roadmap I have in mind; the "...." indicates perhaps more intermediate steps to gain confidence in the new way things work before a major "flip the switch" milestone. This is the roadmap I have in mind; the "...." indicates perhaps more intermediate steps to gain confidence in the new way things work before a major "flip the switch" milestone.
@ -187,7 +187,7 @@ Without "wanted" mutation, there is no `addWantedOutputs`, and (I'm pretty sure)
> An irony of the status quo is we in fact *do* have something called `DrvOutputSubstitutionGoal`! > An irony of the status quo is we in fact *do* have something called `DrvOutputSubstitutionGoal`!
> It just isn't sufficient to get rid of "wanted outputs" from `DerivationGoal`. > It just isn't sufficient to get rid of "wanted outputs" from `DerivationGoal`.
With that fixed....the core of dynamic derivations really is done! With that fixed, the core of dynamic derivations really is done!
There are some usability improvements such as There are some usability improvements such as
- [Nix#8602] Restricted form of "recursive Nix", to give people better incentives / guardrails. - [Nix#8602] Restricted form of "recursive Nix", to give people better incentives / guardrails.