[{"slug":"NixOS-Policy-Breakdown","category":"blog","title":"Political Bikeshedding: NixOS Edition","description":"On Social Dynamics and Leadership","tags":["nix","politics"],"body":"\nThis piece offers a perspective on recent NixOS project challenges from\na long-term contributor. As one of the authors of [RFC 175][175], which\nattempted to address moderation issues but faced obstacles in adoption,\nthe writer brings both experience and a commitment to improving the\nproject's governance.\n\nGiven the complexity of the situation, this article aims to provide a\nhigh-level analysis rather than an exhaustive account. While specific\nexamples are limited for brevity, future pieces may explore more\ndetailed case studies. The current goal is to establish a framework for\nunderstanding and addressing broader issues that are increasingly\nprevalent across the open-source world.\n\n## The Silent Majority and the Myth of \"Community Consensus\"\n\nA significant portion, if not the majority, of people likely despise\npolitics and deliberately disengage from it, focusing on more enjoyable\npursuits. By nature, these individuals aren't necessarily interested in\nhaving their unrelated political or ideological views \"represented\" in\ngroups or forums they join for specific interests, such as hacking.\n\nCrucially, this silent majority challenges the notion of a unified\n\"voice of the community.\" Many claim to speak on behalf of \"the\ncommunity,\" advocating for actions or bans based on supposed community\nconsensus. However, if a silent majority exists, such claims of\nrepresenting the entire community are inherently flawed and potentially\nmisleading, at best.\n\nThe concept of a silent majority and the questionable nature of claimed\ncommunity consensus lead us to examine another critical issue: the\nmisuse of marginalization claims, which many of these voices use as a\nfoundation. Understanding the contextual and temporal nature of\nmarginalization is key to addressing this problem.\n\n## Marginalization is Contextual and Temporal\n\nThe term \"marginal\" has no fixed definition outside a specific context.\nConsider this scenario: someone stands on the far side of a room while\nothers gather at a table. This person detects a threat, perhaps a small\nfire, only visible from their position. They alert the group and come to\nthe table to address the issue. Everyone appreciates their input, and by\njoining the table, they physically become part of the majority.\n\nNow, imagine another fire starts under the table with everyone seated.\nThose at the table, including the previously \"marginal\" individual,\ncan't detect this new threat. Their once unique position is lost, and\nthey're now part of the group that's unaware of the new danger.\n\nIt's crucial to note that even this scenario is relative. To another\ngroup or from a broader perspective, everyone at this table could be\nconsidered marginal. This underscores the importance of context: a\nmarginal position in one setting may be quite common in another.\n\nThis relativity is particularly relevant when considering claims of\nmarginalization within specific communities or projects. Even if\nindividuals are marginalized in broader society, they may hold majority\nor influential positions within a particular project or community. In\nsuch cases, their claims of marginalization within that specific context\nmay not be accurate or relevant.\n\nIn essence, marginalization is a temporary state, not a fixed identity.\nIt's fluid and can shift with changing situations and contexts,\nhighlighting the importance of diverse perspectives and the danger of\nassuming any one group always holds a privileged viewpoint or unique\ninsight in all settings.\n\nThe misuse of marginalization claims has serious consequences.\nIndividuals wield this notion of perpetual marginalization not only to\nspeak for others, but also to justify a degradation of professional\nstandards. This false moral authority has become a shield for behavior\nthat would otherwise be unacceptable, leading us to examine the pitfalls\nof such unchecked conduct.\n\n## The Pitfall of Unchecked Behavior and False Marginalization\n\nTraditionally, public displays of childish behavior were not tolerated\nin professional settings. Recently, however, a troubling trend has\nemerged: the justification of bullying behavior based on claimed\nmarginalized status. This justification often escalates rapidly,\ncreating untenable situations.\n\nCrucially, these individuals are exploiting an identity that lacks a\nconcrete, technical definition. They are not inherently or permanently\nmarginalized; rather, they're hiding behind a facade to maintain special\nprivileges, particularly the ability to \"shout down\" others without\nconsequence.\n\nThis false claim of static marginalization ignores the contextual and\ntemporal nature of marginalization we discussed earlier. It allows\ncertain individuals or groups to maintain a position of perceived moral\nauthority, even when they've become part of, or aligned with the\nmajority. This misuse of claimed status creates an environment where\nbullying is not only tolerated but sometimes even encouraged, as long as\nit comes from the \"right\" sources.\n\nSuch behavior undermines the principles of professionalism, open\ndialogue, and merit-based contribution that should be the hallmarks of\nany healthy community, especially in technical fields. It's essential\nto recognize and address this manipulation to maintain a truly fair and\nproductive environment.\n\n## A Call for Maturity and Productive Incentives\n\nAs an adult and parent, such behavior is more disappointing than\nsurprising. Society might benefit from being more forgiving of mistakes,\nallowing for course correction. In political terms, both sides often\nhave valid points and could learn from each other if they moved past\nsuperficial differences. We should encourage more mature, productive\nmotivations, especially in contexts where many are willing to\ncollaborate constructively.\n\nImportantly, we must consider the role of incentives in shaping\nbehavior. Creatures, including humans, are primarily motivated by\nincentive structures. It's crucial not to inadvertently reward or\nempower those who engage in divisive, derogatory, or unproductive\nbehavior, as this can quickly lead to a self-reinforcing cycle of\nnegative actions.\n\nThankfully, the solution is straightforward: we simply need to\nincentivize civilized behavior. Instead of discouraging constructive\nengagement by labeling it as \"sea lioning\" or \"concern trolling,\" we can\ncultivate an environment that rewards respectful disagreement and\ncollaborative problem-solving irrespective of personal or political\ndifferences.\n\nThe alternative and apparent status quo seems to be a perpetual\nwitch-hunt for an ever-growing list of \"wrong\" opinions. Surely it is\nclear which strategy is more sustainable?\n\n## The Dangers of History Modification\n\nThe core issue lies in social manipulation through selective moderation\nand, crucially, the modification of historical records. When moderation\nteams, often claiming to represent marginalized groups, are empowered to\nalter, delist, or delete past conversations, posts, or decisions, they\ngain the ability to distort the narrative. This practice is, by\ndefinition, a form of rewriting history.\n\nBy condoning or failing to address poor behavior from those claiming\nmarginalized status, these moderators further enable and entrench the\nmisuse of such claims. This creates a self-reinforcing cycle of\nmanipulation and degraded standards, undermining the integrity of\ndiscourse and eroding trust among members.\n\nA relevant example in the Nix project involves Jon Ringer, a\nlong-standing contributor and public figure. Recent controversies have\nportrayed Jon as an instigator and \"mean offensive person.\" However, a\nmore balanced view reveals him as a scapegoat for broader project\ntensions. Crucially, Jon was permanently banned from the project, while\nmany who openly degraded him and made ad hominem attacks on his character\nfaced no consequences. This stark contrast highlights the uneven\napplication of moderation standards.\n\nWhile Jon, like anyone, may have had imperfect moments under extreme\npressure, these pale in comparison to the systematic narrative\nmanipulation by others. The situation exposes a coordinated effort to\ndistort facts, issue threats, and employ bullying tactics to control the\nproject's direction.\n\nThe issue isn't that Jon was never wrong, but that he was consistently\npainted as the primary instigator, regardless of reality. Even when\nheated, Jon generally avoided the name-calling and derogatory behavior\nthat often went unchecked from other parties. His permanent ban, in this\ncontext, underscores the troubling double standard at play in the\nproject's governance.\n\nThis manipulation of context and conversation history not only\nmisrepresents the overall dynamics but also serves to gaslight both\nindividuals and the wider perspective. The impact of this distortion is\nevident in public platforms like Reddit, where observers unfamiliar with\nJon often express views that align with the manipulated narrative. These\ncasual observers, swayed by the dominant portrayal and the absence of\nmeaningful dissenting arguments, tend to perceive Jon as a far greater\nproblem than he actually was.\n\nCrucially, while Jon may have contributed to some tension, he is far from\nthe epicenter of the controversy. In fact, the current issues surrounding\nhim have been brewing for years, consistently instigated by the same\nindividuals who have largely escaped scrutiny as they continue to\nperpetuate divisive narratives.\n\nThe most tangible and regrettable outcome of this scapegoating is that\nthe Nix project has lost a long-standing, highly productive, and\nprofessional contributor. Jon was often very helpful to newcomers, and\nhis departure represents a significant loss to the project. This\nillustrates the real cost of allowing manipulated narratives to drive out\nvaluable members.\n\nSuch power to modify history is dangerous. It allows for the erasure of\ncontext, the silencing of dissenting voices, and the creation of a false\nconsensus. This not only undermines transparency but also erodes trust\nwithin project spaces and misleads those on the periphery. Setting a\nclear precedent against this practice is vital. We must recognize that\nallowing any group to \"clean up\" or selectively edit the historical\nrecord is tantamount to endorsing propaganda[^1]. True professionalism in\nproject management involves facing our history honestly, learning\nfrom it, and moving forward transparently.\n\nSolving these problems requires strong leadership with a commitment to\npreserving the integrity of shared discourse. Leaders must establish\nclear principles that prioritize transparency and resist the temptation\nto sanitize the past. While challenging, this approach is essential for\nmaintaining fairness, fostering genuine progress, and building a\ntrustworthy environment.\n\n## The Solution: Embracing Leadership Principles\n\nThe real solution to Nix's or any project suffering from such childish\nincursion lies in embracing fundamental principles of leadership. Being\na genuinely good leader is challenging. It requires holding oneself to a\nhigher standard than everyone else, and having the courage and conviction\nto guide others to be their best selves, even when they resist.\n\nGood leadership is the only way to be fair to all sides when there is a\ngenuine disagreement. It involves:\n\n1. Setting clear, unambiguous goals and standards of behavior that align\n   with the project's core values. This clarity respects everyone's time,\n   allowing individuals to easily decide whether they align with and wish\n   to participate in the project.\n2. Maintaining transparency and resisting the urge to manipulate\n   historical records.\n3. Fostering respectful, merit-based dialogue while considering the\n   silent majority, not just vocal special interests.\n4. Making decisions based on technical merit and the project's best\n   interests, not personal or ideological biases.\n5. Being willing to address conflicts directly and fairly, without\n   scapegoating individuals or giving special privileges to allies.\n6. Consistently enforcing these standards, making it clear what kind of\n   behavior and contributions are valued in the project.\n\nBy embracing these leadership principles, any project can create an\nenvironment where technical excellence and collaborative spirit thrive.\nIt's a path that requires courage and commitment but offers the best hope\nfor resolving current tensions and preventing future ones.\n\nHowever, implementing these principles requires a conscious choice from\nall contributors, especially from those who have remained silent until\nnow.\n\n## The Great Purge or Professionalism?\n\nThe Nix project faces a critical juncture. A long-standing moderator has\npublicly expressed a desire for a [\"purge\"][purge] of supposed\n\"undesirables.\" This stark reality forces us to confront a fundamental\nchoice: do we embrace professionalism and mutual respect, or do we\nallow divisive, exclusionary behavior to dominate and ultimately derail\nthe entire project?\n\nThis isn't just about Nix; it's a choice many now face. The silent\nmajority, those who typically avoid controversy, may now have to decide\nwhat kind of project space they want to cultivate, and what sort of\nleaders they wish to follow. Inaction is itself a choice; one that may\nlead to the continued erosion of the project's ethic.\n\nWe must ask ourselves: Do we want a forum driven by technical merit and\ncollaborative spirit, or one ruled by ideological purity? The answer to\nthis question will shape the future of Nix and could set a precedent for\nopen-source projects at large.\n\nIt's time for those who value professionalism, open collaboration, and\ntechnical excellence to stand up and be counted. The alternative - an\necosystem stifled by ideological cleansing - is too high a price to pay\nfor our silence.\n\nWhile this piece has focused on Nix, the issues discussed are\nsymptomatic of a growing and worrying trend across the open-source\nworld. Many projects face similar challenges with ideological divisions,\nmanipulated narratives, and the silencing of dissenting voices.\n\nOpen source is far too important to be ruled by narrow-minded and\nexclusionary ideologies. By embracing strong leadership principles and\nfostering environments of mutual respect and professionalism, we can\nensure that open source continues to thrive as a bastion of innovation\nand collaboration.\n\n[175]: https://github.com/NixOS/rfcs/pull/175\n[purge]: https://chaos.social/@hexa/112711384631096150\n\n[^1]: https://en.wikipedia.org/wiki/Historical_negationism\n"},{"slug":"NixOS-Flakes_and_KISS","category":"blog","title":"NixOS, Flakes and KISS","description":"A simpler way to manage the OS Layer","tags":["nix"],"body":"\n## Introduction\n\nThis marks the first post of my very own developer blog, and it comes much later\nthan I had originally anticipated thanks to the ongoing pandemic, coupled with\nsome unforeseen life challenges. My original intent was to start by introducing\nthe concept of Nix Flakes, however, an excellent blog series over at\n[tweag.io](https://www.tweag.io/blog/2020-05-25-flakes) has emerged, expanding\non just that premise. If you are new to flakes, it is highly recommended that\nyou check it out before continuing with this post.\n\nNow, I'd like to introduce a project I've been slowly building up since\nflakes were introduced called [DevOS][DevOS].\n\n# So what is it anyway?\n\nAfter years of working with NixOS, I strongly felt that the community as a whole\ncould benefit from a standardized structure and format for NixOS configurations\nin general. It appears that every developer is  essentially reinventing the\nwheel when it comes to the \"shape\" of their deployments, leading to a lot of\nconfusion as to what the idioms and best practices should be, especially for\nnewcomers.\n\nHaving a mind share to collect the best ideas concerning structure and\nmethod would be valuable, not only for its pragmatic implications, but also to\nhelp ease adoption and onboarding for new NixOS users; something that has\ntraditionally been difficult up to now.\n\nOf course this really hinges on wider community support, as my ideas alone\ndefinitely shouldn't be the final word on what constitutes a correct and well\norganized NixOS codebase. Rather, I am hoping to cajole the community forward\nby providing useful idioms for others to expand on.\n\nEven if my ideas lose out in the end, I sincerely hope they will, at the very\nleast, push the community toward some level of consensus in regards to the way\nNixOS code repositories are structured and managed.\n\nThat said, DevOS appears to be gaining a bit of popularity among new flake\nadopters and I am really quite excited and humbled to see others engage the\nrepository. If you have contributed to the project, thank you so much for your\ntime and support!\n\n# An Arch KISS\n\nI moved over to NixOS after a decades long love affair with Arch Linux. I found\ntheir brand of KISS to be pragmatic and refreshing compared to alternatives\nsuch as Ubuntu or Red Hat. This isn't to dog on those distributions, which I\nalso have used and enjoyed for years, but rather to accentuate my affection\nfor the simplified, and developer focused workflow that Arch Linux enabled for\nmy work stations.\n\nHowever, over the years, I came to resent the several hours of tedious work\nspent doing what amounted to the same small tasks over and over, any time\nissues arose.\n\nMy first attempt to alleviate some of this work was by using Ansible to deploy\nmy common configuration quickly whenever it became necessary. However, I ran\ninto a ton of issues as the Arch repositories updated, and my configurations\ninevitably became stale. Constant, unexpected breakage became a regular\nnuisance.\n\nI then became aware of Nix and NixOS, and hoped that it would live up the\npromise of reproducible system deployment, and after a brief stint of\nprocrastination, I dove head first.\n\n# Great but Not Perfect.\n\nAt first everything seemed almost perfect. NixOS felt like Ansible on steroids,\nand there was more than enough code available in nixpkgs to meet my immediate\nneeds. Getting up to speed on writing derivations and modules was fairly\nstraightforward and the DevOps dream was in sight.\n\nIt wasn't all sunshine and rainbows, as channel updates sometimes caused\nthe same sort of breakage I moved to NixOS to avoid. But simple generation\nrollbacks were a much more welcome interface to this problem than an unbootable\nsystem. It was a measurable improvement from the busy work experienced with Arch. All in all, I felt it was\nwell worth the effort to make the transition.\n\nIt wasn't long before the [rfc][rfcs] that eventually became flakes emerged.\nIt seemed like the solution to many of my few remaining gripes with my\nworkflow. An officially supported and simple way to lock in a specific revision\nof the entire system. No more unexpected and unmanaged breakage!\n\nOf course it took a while for an experimental implementation to arrive, but I\nfound myself digging into the Nix and Nixpkgs PR's to see how flakes worked\nunder the hood.\n\nAround the same time, the ad hoc nature of my NixOS codebase was starting to\nbug at me, and I wanted to try my hand at something more generalized and\ncomposable across machines. I had a first iteration using the traditional\n\"configuration.nix\", but ended up feeling like the whole thing was more\ncomplex than it really needed to be.\n\nMy eagerness to get started using flakes was the perfect excuse to start from\nscratch, and so began DevOS. An attempt to address my concerns, using flakes.\n\n## How does it work?\n\nFirst and foremost, I want to point out that the bulk of the credit goes to the\namazing engineer's who have designed and implemented Nix and the ecosystem\nas a whole over the last decade.\n\nI see a lot of new users struggling to dive in and get up to speed with the Nix\nlanguage, and particularly, getting up and running with a usable and productive\nsystem can take some serious time. I know it did for me.\n\nThe hope for DevOS is to alleviate some of that pain so folks can get to\nwork faster and more efficiently, with less frustration and more enthusiasm for\nthe power that Nix enables. I especially don't want anyone turning away from\nour amazing ecosystem because their onboarding experience was too complex\nor overwhelming.\n\n# Everything is a profile!\n\nAt the heart of DevOS is the [profile][profiles]. Of course, these profiles\nare really nothing more than good ol' NixOS [modules][modules]. The only reason\nI've decided to rebrand them at all is to draw a distinction in how they are\nused. They are kept as simple as possible on purpose; if you understand modules\nyou don't _really_ have anything new to learn.\n\nThe only limitation is that a profile should never declare any new NixOS module\noptions, we can just use regular modules for that elsewhere. Instead, they\nshould be used to encapsulate any configuration which would be useful for more\nthan one specific machine.\n\nTo put it another way, instead of defining my entire NixOS system in a\nmonolithic module, I break it up into smaller, reusable profiles which can\nbe themselves be made up of profiles. Composability is key here, as I don't\nnecessarily want to use every profile on every system I deploy.\n\nAs a concrete example, my [develop][develop], profile pulls in my preferred\ndeveloper tools such as my shell, and text editor configurations. It can be\nthought of as a meta-profile, made up of smaller individual profiles. I can\neither pull in the whole thing, which brings all the dependent profiles along\nwith it, or I can just import a single profile from within, say my zsh\nconfiguration, leaving all the rest unused. Every profile is a directory with\na \"default.nix\" defining it. You can have whatever else you need inside the\nfolder, so long as it is directly related to the profile.\n\nLet's draw the obvious parallel to the Unix philosophy here. Profiles work\nbest when they do one thing, and do it well. Don't provision multiple programs\nin one profile, instead split them up into individual profiles, and then if you\noften use them together, import them both in a parent profile. You can simply\nimport dependent profiles via the \"imports\" attribute as usual, ensuring\neverything required is always present.\n\nThe key is this, by simply taking what we already know, i.e. NixOS modules, and\nsticking to the few simple idioms outlined above, we gain composability and\nreusability without actually having to learn anything new. I want to drill this\npoint home, because that's really all there is to DevOS!\n\nBesides a few simple convenience features outlined below, profiles are the star\nof the show. It's really nothing revolutionary, and that's on purpose! By\nkeeping things simple and organized we gain a level of control and granularity\nwe wouldn't have otherwise without adding real complexity to speak of.\n\n# Really? Everything?\n\nYes! Thanks to built in [home-manager][home-manager] integration, users are\nprofiles, a preferred graphical environment is a profile. Anything that you\ncould imagine being useful on more than one machine is a profile. There are\nplenty of examples available in the \"profiles\" and \"users\" directories, and\nyou can check out my personal \"nrd\" branch, if you want to see how I do things\non my own machines.\n\n# Anything else I should know?\n\nAs mentioned briefly above, DevOS also has some convenience features to make\nlife easier.\n\nFor starters, you might be wondering how we actually define a configuration for\na specific machine. Simple, define the machine specific bits in a nix file\nunder the [hosts][hosts] directory and import any relevant profiles you wish to\nuse from there. The flake will automatically import any nix files in this folder as NixOS\nconfigurations available to build. As a further convenience, the hostname of\nyour system will be set to the filename minus the \".nix\" extension. This makes\nfuture \"nixos-rebuilds\" much easier, as it defaults to looking up your current\nhostname in the flake if you don't specify a configuration to build explicitly.\n\nNow what if we actually just want to define a NixOS module that does declare\nnew NixOS options, you know, the old fashioned way? We'll also want to define\nour own pkgs at some point as well. These are both structured closely to how\nyou might find them in the nixpkgs repository itself. This is so that you can\neasily bring your package or module over to nixpkgs without much modification\nshould you decide it's worth merging upstream.\n\nSo, you'd define a package or module the exact same way you would in nixpkgs\nitself, but instead of adding it to all-packages.nix or module-list.nix, you add\nit to pkgs/default.nix and modules/list.nix. Anything pulled in these two files\nwill become available in any machine defined in the hosts directory, as well as\nto other flakes to import from DevOS!\n\nThis setup serves a dual purpose. For people who already know the nixpkgs\nworkflow, it's business as usual, and for individuals who aren't familiar with\nnixpkgs but wish to become so, they can quickly get up to speed on how to add\npackages and modules themselves, in the exact same way they would do so upstream\nproper.\n\nNow what about overlays? Well, any overlay defined in a nix file under the\noverlays directory will be automatically imported, just as with packages and\nmodules, and are available to all hosts, as well as to other flakes.\n\nWhat if I want to pull a specific package from master instead of from the\nstable release? There is a special file, pkgs/override.nix. Any package listed\nhere will be pulled from nixpkgs unstable rather than the current stable release.\nSimple, easy.\n\nWhat about cachix? It's super easy to add your own cachix link just as you\nwould a regular NixOS configuration. As a bonus, it will be wired up as a flake\noutput so other people can pull in your link directly from your flake! My\npersonal cachix repo is setup by default. It provides the packages the flake\nexports so you don't have to build them.\n\nThat should just about do it for DevOS's current quality of life features, but\nthere are more ideas brewing.\n\n# What's next?\nI'm working on a system for seamlessly importing modules, packages and\noverlays from other flakes, which isn't too hard as it is, but it's messy\nbecause the current \"flake.nix\" has a lot of business logic that gets in the way.\n\nAlso, I would like to start programmatically generating documentation for\neverything. So users can quickly find what goes where and not have to read\ndrawn out blog posts like this to get started. 😛 Nixpkgs is currently\ntransitioning to CommonMark for all documentation, and we will probably follow\nsuite.\n\nAdditionally, I want to implement an easy way to actually install NixOS on the\nbare metal from directly within the project. I know the [deploy-rs][deploy-rs]\nproject is working on this, and I'm interested in supporting their project\nin DevOS so as to add extra flexibility and power to installation and\ndeployment!\n\nAlso, certain parts of the flake should be tested to ensure things don't break.\nWe really have no tests to speak of as is. The auto import functions for the\n\"hosts\" and \"overlays\" directory are good examples.\n\n## A call to arms!\nIf you'd like to help, please jump in. I am very much open to any ideas that\ncould reduce the complexity or simplify the UI. If you have a profile you\nbelieve would be useful to others, please open a [Pull Request][pr].\n\nIf you think I am crazy and wasting my time, please don't hesitate to say so! I\ntypically find critical feedback to be some of the most helpful. Most of all,\nif you made it this far, thanks for taking some time to read about my efforts\nand please consider giving DevOS a shot!\n\n\n[nix]: https://nixos.org\n[DevOS]: https://github.com/divnix/DevOS\n[rfcs]: https://github.com/NixOS/rfcs\n[modules]: https://nixos.org/manual/nixos/stable/index.html#sec-writing-modules\n[profiles]: https://github.com/divnix/devos/tree/template/profiles\n[develop]: https://github.com/divnix/devos/tree/template/profiles/develop\n[hosts]: https://github.com/divnix/devos/tree/template/hosts\n[deploy-rs]:  https://serokell.io/blog/deploy-rs\n[home-manager]: https://github.com/nix-community/home-manager\n[pr]: https://github.com/divnix/devos/pulls\n"},{"slug":"std-action","category":"blog","title":"Standard Action","description":"Do it once, do it right.","tags":["std","nix","devops","github actions"],"body":"\n## CI Should be Simple\n\nAs promised in the [last post](./std), I'd like to expand a bit more on what we've\nbeen working on recently concerning Nix & Standard in CI.\n\nAt work, our current GH action setup is rather _ad hoc_, and the challenge of optimizing that path\naround Nix’s strengths lay largely untapped for nearly a year now. Standard has helped somewhat\nto get things organized, but there has been a ton of room for improvement in the way tasks are\nscheduled and executed in CI.\n\n[Standard Action][action] is our answer. We have taken the last several months of brainstorming\noff and on as time allows, experimenting to find a path that is versatile enough to be useful\nin the general case, yet powerful enough for organizations who need extra capacity. So without\nany further stalling, let's get into it!\n\n## The Gist\n\nThe goal is simple, we want a CI system that only does work once and shares the result from there.\nIf it has been built or evaled before, then we want to share the results from the previous run\nrather than start from scratch.\n\nIt is also useful to have some kind of metadata about our actions, which we can use to build\nmatrices of task runners to accomplish our goals. This also allows us to schedule builds on\nmultiple OS trivially, for example.\n\nTask runners shouldn't have to care about Nix evaluation at all, they should just be able to get\nto work doing whatever they need to do. If they have access to already reified derivations, they\ncan do that.\n\nSo how can we accomplish this? Isolate the evaluation to its own dedicated \"discovery\" phase, and\nshare the resulting /nix/store and a json list describing each task and its target derivations.\n\nFrom there it's just a matter of opimizing the details based on your usecase, and to that end we\nhave a few optional inputs for things like caching and remote building, if you are so inclined.\n\nBut you can do everything straight on the runner too, if you just need the basics.\n\n## How it Works\n\nTalking is fine, but code is better. To that end, feel free to take a look at my own personal CI\nfor my NixOS system and related packages: [nrdxp/nrdos/ci.yml][nrdos].\n\nWhat is actually evaluated during the discovery phase is determined directly in the\n[flake.nix][ci-api].\n\nI am not doing anything fancy here at the moment, just some basic package builds, but that is\nenough to illustrate what's happening. You can get a quick visual by look at the summary of\na given run: [nrdxp/nrdos#3644114900](https://github.com/nrdxp/nrdos/actions/runs/3644114900).\n\nYou could have any number of matrices here, one for publishing OCI images, one for publishing\ndocumentation, one for running deployments against a target environment, etc, etc.\n\nNotice in this particular example that CI exited in 2 minutes. That's because everything\nrepresented by these builds is already cached in the specified action input `cache`, so no work is\nrequired, we simply report that the artifacts already exist and exit quickly.\n\nThere is a run phase that typically starts after this build step which runs the Standard action,\nbut since the \"build\" actions only duty is building, it is also skipped here.\n\nThis is partially enabled by use of the GH action cache. The cache key is set using the following\nformat: [divnix/std-action/discover/action.yml#key][key]. Coupled with the guarantees nix already\ngives us, this is enough to ensure the evaluation will only be used on runners using a matching OS,\non a matching architecture and the exact revision of the current run.\n\nThis is critical for runners to ensure they get an exact cache hit on start, that way they pick\nup where the discovery job left off and begin their build work immediately, acting directly\non their target derivation file instead of doing any more evaluation.\n\n## Caching & Remote Builds\n\nCaching is also a first class citizen, and even in the event that a given task fails (even\ndiscovery itself), any of its nix dependencies built during the process leading up to that failure\nwill be cached, making sure no nix build _or_ evaluation is ever repeated. The user doesn't have\nto set a cache, but if they do, they can be rest assured their results will be well cached, we\nmake a point to cache the entire build time closure, and not just the runtime closure, which is\nimportant for active developement in projects using a shared cache.\n\nThe builds themselves can also be handed off to a more powerful dedicated remote builder. The\naction handles remote builds using the newer and more efficient remote store build API, and when\ncoupled with a special purpose service such as [nixbuild.net](https://nixbuild.net), which your\nauthor is already doing, it becomes incredibly powerful.\n\nTo get started, you can run all your builds directly on the action runner, and if that becomes\na burden, there is a solid path available if and when you need to split out your build phase to a\ndedicated build farm.\n\n## Import from What?\n\nThis next part is a bit of an aside, so feel free to skip, but the process outlined above just so\nhappened to solve an otherwise expensive problem for us at work, outlining how thinking through\nthese problems carefully has helped us improve our process.\n\nIOG in general is a bit unique in the Nix community as one of the few heavy users of Nix’s IFD\nfeature via our [haskell.nix][haskell] project. For those unaware, IFD stands for\n\"import from derivation\" and happens any time the contents of some file from one derivations output\npath is read into another during evaluation, say to read a lock file and generate fetch actions.\n\nThis gives us great power, but comes at a cost, since the evaluator has to stop and build the\nreferenced path if it does not already exist in order to be able to read from it.\n\nFor this reason, this feature is banned from inclusion in nixpkgs, and so the tooling used there\n(Hydra, _et al._) is not necessarily a good fit for projects that do make use of IFD to some extent.\n\nSo what can be done? Many folks would love to improve the performance of the evaluator itself, your\nauthor included. The current Nix evaluator is single threaded, so there is plenty of room for\nsplitting this burden across threads, and especially in the case of IFD, it could theoretically\nspeed things up a great deal.\n\nHowever, improving the evaluator performance itself is actually a bit of a red herring as far as\nwe are concerned here. What we really want to ensure is that we never pay the cost of any given Nix\nworkload more than once, no matter how long it takes. Then we can ensure we are only ever\nbuilding on what has already been done; an additive process if you will. Without careful\nconsideration of this principle beforehand, even a well optimized evaluator would be wasting cycles\ndoing the same evals over and over. There is the nix flake evalulation cache, but it comes with\na few [caveats][4279] on its own and so doesn't currently solve our problem either.\n\nTo give you some numbers, to run a fresh eval of my current project at work takes 35 minutes from a\nclean /nix/store, but with a popullated /nix/store from a previous run it takes only 2.5 minutes.\nSome of the savings is eaten up by data transfer and compression, but the net savings are still\nmassive.\n\nI have already begun brainstorming ways we could elimnate that transfer cost entirely by introducing\nan optional, dedicated [evaluation store](https://github.com/divnix/std-action/issues/10) for those\nwho would benefit from it. With that, there is no transfer cost at all during discovery, and the\nindividual task runners only have to pull the derivations for their particular task, instead of the\nentire /nix/store produced by discovery, saving a ton of time in our case.\n\nEither way, this is a special case optimization, and for those who are content to stick with the\ndefault of using the action cache to share evaluation results, it should more than suffice in the\nmajority of cases.\n\n## Wrap Up\n\nSo essentially, we make due with what we have in terms of eval performance, focus on ensuring we\nnever do the same work twice, and if breakthroughs are made in the Nix evaluator upstream at some\npoint in the future, great, but we don't have to wait around for it, we can minimize our burden\nright now by thinking smart. After all, we are not doing Nix evaluations just for the sake of it,\nbut to get meaningful work done, and doing new and interesting work is always better than repeating\nold tasks because we failed to strategize correctly.\n\nIf we do ever need to migrate to a more complex CI system, these principles themeselves are all\nencapsulated in a few fairly minimal shell scripts and could probably be ported to other\nsystems without incredible effort. Feel free to take a look at the source to see what's really\ngoin on: [divnix/std-action](https://github.com/divnix/std-action).\n\nThere are some places where we could use some [help][7437] from [upstream][2946], but even then, the\nprocess is efficient enough to be a massive improvement, both for my own personal setup, and for\nwork.\n\nAs I mentioned in the previous post though, Standard isn't just about convenience or performance,\nbut arguable the most important aspect is to assist us in being _thorough_. To ensure all\nour tasks are run, all our artifacts are cached and all our images are published is no small feat\nwithout something like Standard to help us automate away the tedium, and thank goodness for that.\n\nFor comments or questions, please feel free to drop by the official Standard [Matrix Room][matrix]\nas well to track progress as it comes in. Until next time...\n\n[action]: https://github.com/divnix/std-action\n[haskell]: https://github.com/input-output-hk/haskell.nix\n[nrdos]: https://github.com/nrdxp/nrdos/blob/master/.github/workflows/ci.yml\n[key]: https://github.com/divnix/std-action/blob/6ed23356cab30bd5c1d957d45404c2accb70e4bd/discover/action.yml#L37\n[7437]: https://github.com/NixOS/nix/issues/7437\n[3946]: https://github.com/NixOS/nix/issues/3946#issuecomment-1344612074\n[4279]: https://github.com/NixOS/nix/issues/4279#issuecomment-1343723345\n[matrix]: https://matrix.to/#/#std-nix:matrix.org\n[ci-api]: https://github.com/nrdxp/nrdos/blob/66149ed7fdb4d4d282cfe798c138cb1745bef008/flake.nix#L66-L68\n"},{"slug":"std","category":"blog","title":"From DevOS to Standard","description":"Why we made Standard, and what it has done for us.","tags":["std","nix","devops"],"body":"\n## Update: A Video is Worth 1000 Blogs\n\nFor those who would rather watch than read, a colleague of mine has whipped up a great video series\nexploring Standard in depth, so drop by the [media secition](../media) for links.\n\n## Two years later...\n\nDevOS started as a fun project to try and get better with Nix and understand this weird new thing\ncalled flakes. Since then and despite their warts, Nix flakes have experienced widespread use, and\nrightfully so, as a mechanism for hermetically evaluating your system & packages that fully locks\nyour inputs and guarantees you some meaningful level of sanity over your artifacts.\n\nYet when I first released it, I never even imagined so many people would find DevOS useful, and I\nhave been truly humbled by all the support and contributions that came entirely spontaneously to the\nproject and ultmately culminated in the current version of [digga][digga], and the divnix org that\nmaintains it.\n\n## Back to Basics\n\nFor whatever reason, it really feels like time to give a brief update of what has come of this\nlittle community experiment, and I'm excited to hopefully clear up some apparent confusion, and\nhopefully properly introduce to the world [Standard](https://github.com/divnix/std).\n\nDevOS was never meant to be an end all be all, but rather a heavily experimental sketch while\nI stumbled along to try and organize my Nix code more effectively. With Standard, we are able to\ndistill the wider experience of some of its contributors, as well as some new friends, and design\nsomething a little more focused and hopefully less magical, while still eliminating a ton of\nboilerplate. Offering both a lightly opinionated way to organize your code into logically typed\nunits, and a mechanism for defining \"standard\" actions over units of the same type.\n\nOther languages make this simple by defining a module mechanism into the language where users are\nfreed from the shackles of decision overload by force, but Nix has no such advantage. Many people\nhoped and even expected flakes to alleviate this burden, but other than the schema Nix expects\nover its outputs, it does nothing to enforce how you can generate those outputs, or how to organize\nthe logical units of code & configuration that generate them.\n\n## A Departure from Tradition\n\nIt is fair to say that the nixpkgs module system has become the sort of \"goto\" means of managing\nconfiguration in the Nix community, and while this may be good at the top-level where a global\nnamespace is sometimes desirable, it doesn't really give us a generic means of sectioning off our\ncode to generate both configuration _and_ derivation outputs quickly.\n\nIn addition to that, the module system is fairly complex and is a bit difficult to anticate the\ncost of ahead of time due to the fixed-point. The infamous \"infinite traces\" that can occur during\na Nix module evaluation almost never point to the actual place in your code where the error\noriginates, and often does even contain a single bit of code from the local repository in the trace.\n\nYet as the only real game in town, the module system has largely \"de facto\" dictated the nature\nof how we organize our Nix code up til now. It lends itself to more of a \"depth first\" approach\nwhere modules can recurse into other modules ad infinitum.\n\n## A Simpler Structure\n\nStandard, in contrast, tries to take an alternative \"breadth first\" approach, ecouraging code\norganization closer to the project root. If true depth is called for, flakes using Standard can\ncompose gracefully with other flakes, whether they use Standard or not.\n\nIt is also entirely unopionated on what you output, there is nothing stopping you from simply\nexporting NixOS modules themselves, for example, giving you a nice language level\ncompartmentalization strategy to help manager your NixOS, Home Manager or Nix Darwin configurations.\n\nAdvanced users may even write their own types, or even extend the officially supported ones. We\nwill expand more on this in a later post.\n\nBut in simple terms, why should we bother writing the same script logic over and over when we can be\nguaranteed to recieve an output of a specific type, which guarantees any actions we define for the\ntype at large will work for us: be it deploying container images, publishing sites, running\ndeployments, or invoking tests & builds.\n\nWe can ensure that each image, site, or deployment is tested, built, deployed and published in\na sane and well-defined way, universally. In this way, Standard is meant to not only be convenient,\nbut comprehensive, which is an important property to maintain when codebases grow to non-trivial\nsize.\n\nThere is also no fixed-point so, anecdotably, I have yet to hit an eval error in Standard based\nprojects that I couldn't quickly track down; try saying that about the module system.\n\n## A CLI for productivity\n\nThe Nix cli can sometimes feel a little opaque and low-level. It isn't always the best interface\nto explain and explore what we can actually _do_ with a given project. To address this issue in\na minimal and clean way, we package a small go based cli/tui combo to quickly answer exactly this\nquestion, \"What can I do with this project?\".\n\nThis interface is entirely optional, but also highly useful and really rather trivial thanks to a\npredicatable structure and well typed outputs given to us in the Nix code. The schema for anything\nyou can do follows the same pattern: \"std //$cell/$block/$target:$action\". Here the \"cell\" is the\nhighest level \"unit\", or collection of \"blocks\", which are well-typed attribute sets of \"targets\"\nsharing a colleciton of common \"actions\" which can be performed over them.\n\n### At a Glance\n\nThe TUI is invaluable for quickly getting up to speed with what's available:\n\n```console\n┌────────────────────────────────────────────────────────────────────────────────┐┌───────────────────────────────────┐\n│|  Target                                                                       ││   Actions                         │\n│                                                                                ││                                   │\n│  176 items                                                                     │││ build                            │\n│                                                                                │││ build this target                │\n│  //automation/packages/retesteth                                               ││                                   │\n│  testeth via RPC. Test run, generation by t8ntool protocol                     ││  run                              │\n│                                                                                ││  exec this target                 │\n││ //automation/jobs/cardano-db-sync                                             ││                                   │\n││ Run a local cardano-db-sync against our testnet                               ││                                   │\n│                                                                                ││                                   │\n│  //automation/jobs/cardano-node                                                ││                                   │\n│  Run a local cardano-node against our testnet                                  ││                                   │\n│                                                                                ││                                   │\n\n```\n\n## A Concise Show & Tell\n\nThe central component of Standard is the cell block API. The heirarchy is \"cell\"→\"block\", where\nwe defined the individual block types and names directly in the flake.nix.\n\nThe function calls in the \"cellBlocks\" list below are the way in which we determine which \"actions\"\ncan be run over the contents of the given block.\n\n```nix\n# flake.nix\n{\n  inputs.std.url = \"github:divnix/std\";\n  outputs = inputs: inputs.std.growOn {\n    inherit inputs;\n    systems = [\"x86_64-linux\"];\n    # Every file in here should be a directory, that's your \"cell\"\n    cellsFrom = ./nix;\n    # block API declaration\n    cellBlocks = [\n      (std.functions \"lib\")\n      (std.installables \"packages\")\n      (std.devshells \"devshells\")\n    ];\n  };\n}\n\n# ./nix/dev/packages.nix\n# nix build .#$system.dev.packages.project\n# std //dev/packages/project:build\n{\n  inputs, # flake inputs with the `system` abstracted, but still exposed when required\n  cell # reference to access other blocks in this cell\n}: let\n  inherit (inputs.nixpkgs) pkgs;\nin\n{\n  project = pkgs.stdenv.mkDerivation {\n    # ...\n  };\n}\n\n# ./nix/automation/devshells/default.nix\n# nix develop .#$system.dev.devshells.dev\n# std //automation/devshells/dev:enter\n{\n  inputs,\n  cell\n}: let\n  inherit (inputs) nixpkgs std;\n  inherit (nixpkgs) pkgs;\n  # a reference to other cells in the project\n  inherit (inputs.cells) packages;\nin\n{\n  dev = std.mkShell { packages = [packages.project]; };\n}\n```\n\n## Encouraging Cooperation\n\nStandard has also given us a useful mechanism for contributing back to upstream where it makes\nsense. We are all about maintaining well-defined boundaries, and we don't want to reimplement the\nworld if the problem would be better solved elsewhere. Work on Standard has already led to several\nuseful contributions to both nixpkgs and even a few in nix proper, as well as some in tangentially\nrelated codebases, such as github actions and go libraries.\n\nOne very exciting example of this cooperation is the effort we've expended integrating\n[nix2container][n2c] with Standard. The work has given us insights and position to begin defining an\nofficially supported specification for [OCI images][oci] built and run from Nix store paths, which\nis something that would be a huge win for developers everywhere!\n\nWe believe interoperability with existing standards is how Nix can ultimately cement itself into\nthe mainstream, and in a way that is unoffensive and purely additive.\n\n## CI simplified\n\nInstead of making this a mega post, I'll just leave this as a bit of a teaser for a follow-up post\nwhich will explore our recent efforts to bring the benefits Standard to GitHub Actions _a la_\n[std-action][action]. The target is a Nix CI system that avoids ever doing the same work more than\nonce, whether its evaluating or building, and versatile enough to work from a single user project\nall the way up to a large organization's monorepo. Stay tuned...\n\n[digga]: https://github.com/divnix/digga\n[nosys]: https://github.com/divnix/nosys\n[action]: https://github.com/divnix/std-action \n[grow]: https://std.divnix.com/guides/growing-cells.html\n[harvest]: https://github.com/divnix/std/blob/main/src/harvest.nix\n[n2c]: https://github.com/nlewo/nix2container\n[oci]: https://github.com/opencontainers/image-spec/issues/922\n"}]