feat: switch to minimal zola-based apollo theme
This commit is contained in:
Timothy DeHerrera
4
content/_index.md
Normal file
4
content/_index.md
Normal file
@@ -0,0 +1,4 @@
|
||||
+++
|
||||
[extra]
|
||||
section_path = "blog/_index.md"
|
||||
+++
|
||||
14
content/about.md
Normal file
14
content/about.md
Normal file
@@ -0,0 +1,14 @@
|
||||
+++
|
||||
title = "About"
|
||||
path = "about"
|
||||
+++
|
||||
|
||||
Tim DeHerrera (nrdxp).
|
||||
|
||||
Engineer at [IOG](https://iog.io) developing interoperable blockchain infrastructure for the Cardano ecosystem.
|
||||
|
||||
Co-founder of the [Ekala Project](https://github.com/ekala-project) - next-gen tooling for reproducible builds and declarative systems.
|
||||
|
||||
Specializing in distributed systems, build infrastructure, and network optimization. I am a polyglot systems programmer with a decade of experience, crafting tools that emphasize reproducibility and determinism. My engineering approach favors expressive type systems and functional programming paradigms, with current emphasis on Rust and Nix ecosystems.
|
||||
|
||||
gpg: [5471 8D2B 78DC AA9C 7702 96F1 8985 725D B5B0 C122](https://github.com/nrdxp.gpg)
|
||||
130
content/blog/12-years.md
Normal file
130
content/blog/12-years.md
Normal file
@@ -0,0 +1,130 @@
|
||||
---
|
||||
title: "Twelve Years as a Digital Hermit"
|
||||
description: "Why I left Social Media a Over a Decade Ago, and Why I Now Return"
|
||||
taxonomies:
|
||||
tags:
|
||||
- ramblings
|
||||
- propaganda
|
||||
- philosophy
|
||||
author: "Tim D"
|
||||
authorGithub: "nrdxp"
|
||||
authorImage: "https://avatars.githubusercontent.com/u/34083928?v=4"
|
||||
authorTwitter: "nrdexp"
|
||||
date: "2024-10-11"
|
||||
category: "personal"
|
||||
---
|
||||
|
||||
**Reader Advisory:** This post is sarcastic, and a bit raw. Those without a sense of humor, or an aversion to hard facts, venture no further. Also, these opinions are my own; subject to change, etc, etc.
|
||||
|
||||
*(not that I should have to say any of that to grown adults, imho, but alas...)*
|
||||
|
||||
## My Life the Contradiction
|
||||
|
||||
It has been confusing, even for me at times, to live my life being fascinated with computer technology from a very early age, yet having nothing to do with major social media platforms since around 2012 - even as the rest of the world became utterly infatuated with it, apparently seeing it as some great technical marvel (lol).
|
||||
|
||||
From being an obsessive gamer as a child, breaking games with my GameShark, to studying topics no 10-year-old has any business studying (denotational semantics, anyone?), to building my own machines and making stupid little HTML websites, hacking MySpace and feeling proud, and just generally being the neighborhood's goto free IT support (when people still spoke with their neighbors).
|
||||
|
||||
Then there was trying to teach myself C++ at 13, which I gave up, for reasons that should be obvious to those still clutching to some sanity. Later, due to less than favorable economic conditions and a mountain of responsibility in my early adulthood, having children at a young age, I lost hope that I might ever do anything professionally with technology, working whatever job I could find, for whatever wage they would pay. My personal favorite was digging through folks' garbage for the purpose of "recycling".
|
||||
|
||||
Still, I couldn't resist tinkering, breaking, modding, ripping, hosting, or hacking anything within reach during whatever free time I had. Before Spotify made such efforts redundant, I had built quite a respectable music library. Don't arrest me for using torrents - I know, I know. I couldn't afford to pay for all that music, though I did try to actually purchase albums when I could.
|
||||
|
||||
I remain forever indebted to countless artists in that collection for keeping my hopes alive through those trying and difficult years, dim as they were. I still hope someday I can do something cool to pay them back (music blockchain, anyone?). And let's not forget about the endless - and I mean ENDLESS - Android ROM hacks in the early days. Good times...
|
||||
|
||||
In retrospect, I suppose it was only natural that I would eventually build up enough hands-on experience to do this professionally, but that wasn't really my interest. I was enthralled - and in many ways still am - with the machine itself and what it made possible. How we managed to create these things in the first place remains a great mystery to me. No matter how much I learn, there's always more to know. It's the dream life of that never-ending 10-year-old uber-nerd still living inside me.
|
||||
|
||||
This passion is even expressed through my handle, nrdxp. I'm always looking to gain more experience, to reach the next level, to build something awesome and useful. I think there were a lot of kids like me in the 90s, but I don't think all of them made it this far, and for that miracle alone, I'll always be grateful. That said, there's still a long way to go - growing up poor and starting a career late (with 3 children to boot) has severe and long-lasting consequences.
|
||||
|
||||
For one thing, I don't own a home. I've never lived in a home owned by anyone in my family. In that sense, I guess I've always felt a bit "homeless." When I first started my professional career, I had some hope that I could save and get one - well, then COVID happened and brought me back to reality real quick. Why mention this? Because as grateful as I am, I want to make it clear that I'm not some oracle from on high. I still have struggles, battles, and demons to overcome - things so laughably and persistently bothersome that they're hardly worth mentioning. I'm not quite certain if I'll ever fully overcome them, but I'll certainly continue trying. What else is there? Complain? Yeah, what good does that do? Gross...
|
||||
|
||||
## Not an Innovation; a Virus
|
||||
|
||||
Speaking of complaints and societal problems, this leads nicely into my next point - the very reason I stepped away from the digital chaos in the first place.
|
||||
|
||||
Why did I abandon all forms of social media in 2012, deleting my Facebook (still dominant at the time), and never turning back? Well, as fascinated as I am with technology, social media is quite honestly just boring to me - but it's worse than that. At first, I had some hope that this new thing could actually help usher in a new era of connectedness (naive, I know - I was young), and I tried to push for it.
|
||||
|
||||
I used social media like you're not supposed to, apparently: actually trying to have conversations while maintaining dignity and respect (how stupid), while still pressing people on their quite obviously dogmatic and rigid beliefs. I used to believe that people were just ignorant, that if only they would come to see the light, they would understand. I see a lot of folks still running around like that. All I can say is... good luck.
|
||||
|
||||
But from what I've learned now, it isn't true. People like their delusions - they use them as a shield. To some degree, I believe we all do. In that sense, social media has been nothing but a "delusion amplification engine" for society. For the youngsters out there unable to remember a world before the internet, it probably isn't as jarring. They probably think we were all running around hanging each other over our race, where we choose to stick our genitals, and every other trivial difference - and that somehow, the exposure of social media saved us from all that bigotry.
|
||||
|
||||
But my experience was quite the opposite. Growing up in a Latino family, I can't remember a single instance - not even one - where race was an issue or was made an issue. It's quite clear to me now, seeing the juxtaposition of that time (ah, the 90s) to ours, that that was the proper way to handle trivial differences: ignore them. They don't matter. Really, seriously, they don't fucking matter. Be a bit more grateful that you're still above ground; you won't always be. Find a reason to appreciate this moment instead of condemning it, or you know, don't - whatever, enjoy your bitterness.
|
||||
|
||||
And of course, the government would have you believe they're on your side. They want to end all the "isms" of the world, don't ya know? I'm sure that's why they keep such exacting stats on everything, never shut up about it, and shove it down your throat at every opportunity and on every single form you've ever filled out.
|
||||
|
||||
You wanna know my race or ethnicity to "track" progress? Haha, yeah right. Read a history book, buddy - I know better. Of course, it's not just big daddy Gov's fault. Indeed, these large tech firms have become nearly indistinguishable from governments in some ways, trying to enforce all kinds of arbitrary rules for the sake of nothing but ~their own profit~ "the good of the people".
|
||||
|
||||
But therein lies something close to the core issue for me: governments and private companies alike are *both* quite incompetent at defining and upholding moral behavior. The former simply has no incentive but to continue growing, even beyond all sense or reason, until everything you do - even taking a breath - is illegal and, more importantly, taxable. The latter will change their so-called "values" on a dime to appease the masses, quite happy to talk down to you from both angles.
|
||||
|
||||
And what has social media given us then? Global mob rule; utter stupidity so obvious even children can see it, yet nobody dares address it for fear of facing the mob. Guess what? A mob of rabid idiots on the internet isn't actually that scary anyway. What are they gonna do? Sure, there are the psychos doxxing and swatting and all that nonsense, but psychos have always been around - they're just the canary in the coal mine for a much larger issue - one that strikes at the very heart of our modern delusions.
|
||||
|
||||
|
||||
## No Right to Safety; The New Religion
|
||||
|
||||
This leads into what might be my most controversial point, which I'm sure will mark me as universally hated inside the delusion engine, not to mention the new and improved delusion generation engine 2.0 (incorrectly known as AI). You have no right to safety in society - anybody who promises it to you is either lying to take something from you or delusional about their own capacity to keep even themselves safe. That's not to say you have no right to defend yourself - you certainly do. Ironically enough, big daddy GovTech doesn't think you should, because they "know better." Gee, I wonder why?
|
||||
|
||||
But I digress - it is reality itself, not law, governments, or companies, which dictates that this world is inherently unsafe. There is no permanent fix. A "perfectly safe" space would be nearly indistinguishable from a prison cell, and even then, a big meteor could ruin your day anytime. Or did you already forget that we're all going to die someday? Maybe you believe we'll be uploaded to the machine instead and live forever, to which I would ask: How is this not religious?
|
||||
|
||||
What empirical evidence exists that such a thing is even possible? But this is the new religion - giving up anything and everything, our very thoughts themselves, for the "right" to "blend in," to "be safe," to be on the "right side of history" so we can survive long enough to experience the grand transhuman utopia.
|
||||
|
||||
Anybody who has read a history book knows exactly what side modern governments and tech firms are on, and anybody - well-intentioned or not - who feeds into it is a potential danger. Remember The Matrix? I know it's been done to death, but there's still one line in particular that rings even truer today than when it was made: "anyone who has not been unplugged is a potential agent."
|
||||
|
||||
If you don't stand up, put on your big boy pants, and say "No thanks, I don't need your advice on how to speak, and certainly not on how to think - lmao, are you serious? The arrogance..." - then it's only a matter of time before you simply go along to get along, and shove anyone down who tries to call you out. This is why it's so crucial to maintain the courage to speak your mind.
|
||||
|
||||
The mind, and especially your ability to express your true thoughts earnestly, is like a muscle. If you don't exercise it, it will atrophy fairly quickly, until you can no longer think at all. All you can do is scroll for your daily dose of "what to think."
|
||||
|
||||
## The Forgotten Utility of Ridicule
|
||||
|
||||
So what's the antidote to this mindless conformity? What weapon do we have against this tide of enforced groupthink? Ridicule is the proper response - the header says it all.
|
||||
|
||||
This kind of bullshit morality posturing, colloquially known as "wokeness," is worthy of any and all forms of human ridicule. And no, not just to be mean or vent. Ironically, after mentioning how impossible it is to "be safe" in this world, there are certainly degrees to which one's safety can be practically improved, and ridiculing ridiculous tyrants who would have you kissing their feet is one sure-fire way to do it.
|
||||
|
||||
No, you're not being compassionate, no you don't fucking care about their genitals - I think we all universally agree that "our" genitals are the most important, right? Stop pretending, fuck the show. You can't make me, really and truly. Call me an asshole - actually, I kinda am - but one thing I won't do, even when you want me to, is hate. Hate is the corrosion that led us to this dismal position in the first place. Yoda taught us these principles years ago people, come on.
|
||||
|
||||
What I will do, however, is unapologetically ridicule the ridiculous. Go figure...
|
||||
|
||||
In all seriousness, we live in an era ruled by bitterness and delusion. And in such a short time - a generation and a half - from likely the least racist society that has ever been (the 90s children) to today; all because *you* won't stand up and say what we all know you likely think. The only people who actually believe any of the modern rhetoric are parrots who gave up their right to think independently long ago. As was said by a far more brilliant man (gendered speech, OMG!) than myself, "it takes someone of great intelligence to come up with ideas so colossally and utterly ***stupid***."
|
||||
|
||||
Don't believe for a second that just because someone has some fancy pieces of paper on a wall that they have any right - even a little - to tell you how to think. In fact, that's a sign of incompetence in my book. Real masters of any craft should be able to trivially *demonstrate* what is true and correct. If you have to force it down my throat, perhaps you're just a bit insecure about your own abilities; just maybe?
|
||||
|
||||
Another one to remember (thanks Batman): "you will never be ridiculed by someone doing more than you." In other words, if someone thinks less of you, who cares? It's just a sign they probably feel like you're ahead of them in some way, and they're simply trying to pull you back down. Take it as a compliment and move on.
|
||||
|
||||
Paper means nothing. Like I said, the brain is a muscle - doesn't matter if you were a world-class bodybuilder with all the trophies to boot, if you stop working, you start dying. Yet we live in a world where we take all our orders, *all* our opinions, spoon-fed from idiots with fancy papers, which are dutifully repeated over and over by every "influencer" and their grandma (yes, grandma has a channel now too).
|
||||
|
||||
You want to influence something real? Think for your damn self. I know, it's scary after all this time, but if you really give a damn about "staying safe," it's your best bet. Or don't, and sit comfortable in your delusion until the day comes that you're no longer part of the acceptable and "righteous" crowd. That day is coming - in fact, for very many it's already here.
|
||||
|
||||
But I digress - where we're heading now is anything but "safe." Seriously, I implore you, it may sound sarcastic (and it is a bit) but read a history book. Just pick one. All this nonsense, all this rhetoric has happened before; many times. Just pick an era at random, and I'm quite confident you'll find some form of it or another. Perhaps all this madness is inevitable, but I don't actually care...
|
||||
|
||||
I am one man, living my one life, and I don't want to be "told" by anyone - least of all a brainless mob so utterly afraid to think out of line that they share notes before every public statement - how to think or what to feel, or when or how to speak. Again, no thanks, I got this. I'll take responsibility for my mistakes and take credit for my own wins. I don't need your help, not at any cost, not even if I have to build everything I work on alone, completely in isolation. Where's the inclusivity for folks like me, exactly?
|
||||
|
||||
And the definitions of unacceptable just keep expanding, as they do and have (I'm telling you - history book). To some, it's a crime merely to exist with the wrong genitals, because that's progress somehow...
|
||||
|
||||
This segment could go on and on, but I'll just remind you of the header. There is great social utility in ridicule. It's powerful - it keeps people safe from these idiotic ideas that actually convince them to permanently deform themselves for the sake of approval.
|
||||
|
||||
But that's just the thing: Identity isn't a feeling, it's defined by your character. How you look or what cosmetic appearance you possess will do little to affect your character, and in fact, focusing on it too much will likely degrade it, seeing as it's totally irrelevant in the first place...
|
||||
|
||||
## Wait Why, Again?
|
||||
|
||||
After all this criticism and heated commentary, you might wonder why I'm back on social media at all. So yeah, this got heated in parts, but I honestly feel it is necessary at this point. Years of watching from the sidelines, trying to be "considerate," have only confirmed what I suspected - silence changes nothing.
|
||||
|
||||
And for what was I silent? To get ridiculed and excluded anyway because I refuse to give up my right to think to the great "Constitutional Assembly" of delusion and apathy?
|
||||
|
||||
"And as my first act, I appoint all my friends to the board ~of directors~ to defend ~the empire~ democracy and ~my vested-interests~ the goodwill of the community!"
|
||||
|
||||
And to be clear, I still think social media is a toxic, narcissistic breeding ground for idiocy and delusion, largely responsible for this record-breaking societal decline, distracting us all the while, like the great propaganda masters of television could have only dreamed. Yet I'm back here on X... why?
|
||||
|
||||
Maybe I just want a little attention like everyone else? You can argue that if you like, but I've been doing just fine on my own for quite some time now. If anything, the little bit of attention I've attracted so far has been an annoyance and a burden. It also feels dangerous somehow. But here I am... somewhat because of that last section - there aren't enough folks with the courage to ridicule the ridiculous, so I will, come what may (at least while it's still "legal" in my country; damn free speech...).
|
||||
|
||||
But it's not just that. I'm older now, and I know there are still younger folks who don't have the experience I've gained. Quite frankly, they seem to need an example of what courage looks like. No, it isn't perfect, it's not ripped to shreds with perfect abs, either. It is literally just saying what you really think - damn the consequences.
|
||||
|
||||
So anyway, there you go zealots, all the ammo you'll ever need to condemn me. Consider it a free gift, but know that there's plenty more where that came from.
|
||||
|
||||
Countries are not supposed to give in to terrorism (whatever the fuck that means), and I will not give in to precious bullies with fragile little egos. Sorry, but even if this generation is largely toasted, someone still has to set an example for the next one, however imperfectly, or things will continue to get worse. I guess this is a sentiment only parents really understand, and there are so few parents nowadays (sigh).
|
||||
|
||||
I digress though - I may dislike it (despise it even), but social media is the only playground anyone wants to play on. It's the only place to make any sort of stand, like it or not.
|
||||
|
||||
And I really want to stress this: the most important lesson I've learned from my study of history is that it can, and will *always* get worse, unless and until somebody says, "No, that's enough. I'm quite finished with your little tyranny, thank you - hang me if you must."
|
||||
|
||||
And so I'll end on that note. Hang me if you must... As much as I have always loved technology - I wouldn't even know what to do with myself if I didn't do what I'm doing now - I absolutely and positively *refuse* to perpetuate or feed into this nonsense, even a little bit.
|
||||
|
||||
If the open-source ecosystem had half the spirit of the pioneers who founded it, we would be on the front lines against such rabid propaganda. But instead, we stand quietly on the sidelines, at best asking questions that go perpetually ignored by the zealots, all the while apologizing for our intrusive thoughts.
|
||||
|
||||
Now, if you've made it this far, stay tuned - I've got some things coming down the pipe to try and revitalize a once-thriving world of thoughts and ideas, free from the burden of zealots and thought police. Until then, take care of yourself, and don't be afraid to use that beautiful jelly inside your skull, before you lose it!
|
||||
297
content/blog/NixOS Policy Breakdown.md
Normal file
297
content/blog/NixOS Policy Breakdown.md
Normal file
@@ -0,0 +1,297 @@
|
||||
---
|
||||
title: "Political Bikeshedding: NixOS Edition"
|
||||
description: On Social Dynamics and Leadership
|
||||
taxonomies:
|
||||
tags:
|
||||
- nix
|
||||
- politics
|
||||
author: Tim D
|
||||
authorGithub: nrdxp
|
||||
authorImage: https://avatars.githubusercontent.com/u/34083928?v=4
|
||||
authorTwitter: nrdxp52262
|
||||
date: "2024-07-02"
|
||||
category: politics
|
||||
---
|
||||
|
||||
This piece offers a perspective on recent NixOS project challenges from
|
||||
a long-term contributor. As one of the authors of [RFC 175][175], which
|
||||
attempted to address moderation issues but faced obstacles in adoption,
|
||||
the writer brings both experience and a commitment to improving the
|
||||
project's governance.
|
||||
|
||||
Given the complexity of the situation, this article aims to provide a
|
||||
high-level analysis rather than an exhaustive account. While specific
|
||||
examples are limited for brevity, future pieces may explore more
|
||||
detailed case studies. The current goal is to establish a framework for
|
||||
understanding and addressing broader issues that are increasingly
|
||||
prevalent across the open-source world.
|
||||
|
||||
## The Silent Majority and the Myth of "Community Consensus"
|
||||
|
||||
A significant portion, if not the majority, of people likely despise
|
||||
politics and deliberately disengage from it, focusing on more enjoyable
|
||||
pursuits. By nature, these individuals aren't necessarily interested in
|
||||
having their unrelated political or ideological views "represented" in
|
||||
groups or forums they join for specific interests, such as hacking.
|
||||
|
||||
Crucially, this silent majority challenges the notion of a unified
|
||||
"voice of the community." Many claim to speak on behalf of "the
|
||||
community," advocating for actions or bans based on supposed community
|
||||
consensus. However, if a silent majority exists, such claims of
|
||||
representing the entire community are inherently flawed and potentially
|
||||
misleading, at best.
|
||||
|
||||
The concept of a silent majority and the questionable nature of claimed
|
||||
community consensus lead us to examine another critical issue: the
|
||||
misuse of marginalization claims, which many of these voices use as a
|
||||
foundation. Understanding the contextual and temporal nature of
|
||||
marginalization is key to addressing this problem.
|
||||
|
||||
## Marginalization is Contextual and Temporal
|
||||
|
||||
The term "marginal" has no fixed definition outside a specific context.
|
||||
Consider this scenario: someone stands on the far side of a room while
|
||||
others gather at a table. This person detects a threat, perhaps a small
|
||||
fire, only visible from their position. They alert the group and come to
|
||||
the table to address the issue. Everyone appreciates their input, and by
|
||||
joining the table, they physically become part of the majority.
|
||||
|
||||
Now, imagine another fire starts under the table with everyone seated.
|
||||
Those at the table, including the previously "marginal" individual,
|
||||
can't detect this new threat. Their once unique position is lost, and
|
||||
they're now part of the group that's unaware of the new danger.
|
||||
|
||||
It's crucial to note that even this scenario is relative. To another
|
||||
group or from a broader perspective, everyone at this table could be
|
||||
considered marginal. This underscores the importance of context: a
|
||||
marginal position in one setting may be quite common in another.
|
||||
|
||||
This relativity is particularly relevant when considering claims of
|
||||
marginalization within specific communities or projects. Even if
|
||||
individuals are marginalized in broader society, they may hold majority
|
||||
or influential positions within a particular project or community. In
|
||||
such cases, their claims of marginalization within that specific context
|
||||
may not be accurate or relevant.
|
||||
|
||||
In essence, marginalization is a temporary state, not a fixed identity.
|
||||
It's fluid and can shift with changing situations and contexts,
|
||||
highlighting the importance of diverse perspectives and the danger of
|
||||
assuming any *one* group always holds a privileged viewpoint or unique
|
||||
insight in all settings.
|
||||
|
||||
The misuse of marginalization claims has serious consequences.
|
||||
Individuals wield this notion of perpetual marginalization not only to
|
||||
speak for others, but also to justify a degradation of professional
|
||||
standards. This false moral authority has become a shield for behavior
|
||||
that would otherwise be unacceptable, leading us to examine the pitfalls
|
||||
of such unchecked conduct.
|
||||
|
||||
## The Pitfall of Unchecked Behavior and False Marginalization
|
||||
|
||||
Traditionally, public displays of childish behavior were not tolerated
|
||||
in professional settings. Recently, however, a troubling trend has
|
||||
emerged: the justification of bullying behavior based on claimed
|
||||
marginalized status. This justification often escalates rapidly,
|
||||
creating untenable situations.
|
||||
|
||||
Crucially, these individuals are exploiting an identity that lacks a
|
||||
concrete, technical definition. They are not inherently or permanently
|
||||
marginalized; rather, they're hiding behind a facade to maintain special
|
||||
privileges, particularly the ability to "shout down" others without
|
||||
consequence.
|
||||
|
||||
This false claim of static marginalization ignores the contextual and
|
||||
temporal nature of marginalization we discussed earlier. It allows
|
||||
certain individuals or groups to maintain a position of perceived moral
|
||||
authority, even when they've become part of, or aligned with the
|
||||
majority. This misuse of claimed status creates an environment where
|
||||
bullying is not only tolerated but sometimes even encouraged, as long as
|
||||
it comes from the "right" sources.
|
||||
|
||||
Such behavior undermines the principles of professionalism, open
|
||||
dialogue, and merit-based contribution that should be the hallmarks of
|
||||
any healthy community, especially in technical fields. It's essential
|
||||
to recognize and address this manipulation to maintain a truly fair and
|
||||
productive environment.
|
||||
|
||||
## A Call for Maturity and Productive Incentives
|
||||
|
||||
As an adult and parent, such behavior is more disappointing than
|
||||
surprising. Society might benefit from being more forgiving of mistakes,
|
||||
allowing for course correction. In political terms, both sides often
|
||||
have valid points and could learn from each other if they moved past
|
||||
superficial differences. We should encourage more mature, productive
|
||||
motivations, especially in contexts where many are willing to
|
||||
collaborate constructively.
|
||||
|
||||
Importantly, we must consider the role of incentives in shaping
|
||||
behavior. Creatures, including humans, are primarily motivated by
|
||||
incentive structures. It's crucial not to inadvertently reward or
|
||||
empower those who engage in divisive, derogatory, or unproductive
|
||||
behavior, as this can quickly lead to a self-reinforcing cycle of
|
||||
negative actions.
|
||||
|
||||
Thankfully, the solution is straightforward: we simply need to
|
||||
incentivize civilized behavior. Instead of discouraging constructive
|
||||
engagement by labeling it as "sea lioning" or "concern trolling," we can
|
||||
cultivate an environment that rewards respectful disagreement and
|
||||
collaborative problem-solving irrespective of personal or political
|
||||
differences.
|
||||
|
||||
The alternative and apparent *status quo* seems to be a perpetual
|
||||
witch-hunt for an ever-growing list of "wrong" opinions. Surely it is
|
||||
clear which strategy is more sustainable?
|
||||
|
||||
## The Dangers of History Modification
|
||||
|
||||
The core issue lies in social manipulation through selective moderation
|
||||
and, crucially, the modification of historical records. When moderation
|
||||
teams, often claiming to represent marginalized groups, are empowered to
|
||||
alter, delist, or delete past conversations, posts, or decisions, they
|
||||
gain the ability to distort the narrative. This practice is, by
|
||||
definition, a form of rewriting history.
|
||||
|
||||
By condoning or failing to address poor behavior from those claiming
|
||||
marginalized status, these moderators further enable and entrench the
|
||||
misuse of such claims. This creates a self-reinforcing cycle of
|
||||
manipulation and degraded standards, undermining the integrity of
|
||||
discourse and eroding trust among members.
|
||||
|
||||
A relevant example in the Nix project involves Jon Ringer, a
|
||||
long-standing contributor and public figure. Recent controversies have
|
||||
portrayed Jon as an instigator and "mean offensive person." However, a
|
||||
more balanced view reveals him as a scapegoat for broader project
|
||||
tensions. Crucially, Jon was permanently banned from the project, while
|
||||
many who openly degraded him and made *ad hominem* attacks on his
|
||||
character faced no consequences. This stark contrast highlights the
|
||||
uneven application of moderation standards.
|
||||
|
||||
While Jon, like anyone, may have had imperfect moments under extreme
|
||||
pressure, these pale in comparison to the systematic narrative
|
||||
manipulation by others. The situation exposes a coordinated effort to
|
||||
distort facts, issue threats, and employ bullying tactics to control the
|
||||
project's direction.
|
||||
|
||||
The issue isn't that Jon was never wrong, but that he was consistently
|
||||
painted as the primary instigator, regardless of reality. Even when
|
||||
heated, Jon generally avoided the name-calling and derogatory behavior
|
||||
that often went unchecked from other parties. His permanent ban, in this
|
||||
context, underscores the troubling double standard at play in the
|
||||
project's governance.
|
||||
|
||||
This manipulation of context and conversation history not only
|
||||
misrepresents the overall dynamics but also serves to gaslight both
|
||||
individuals and the wider perspective. The impact of this distortion is
|
||||
evident in public platforms like Reddit, where observers unfamiliar with
|
||||
Jon often express views that align with the manipulated narrative. These
|
||||
casual observers, swayed by the dominant portrayal and the absence of
|
||||
meaningful dissenting arguments, tend to perceive Jon as a far greater
|
||||
problem than he actually was.
|
||||
|
||||
Crucially, while Jon may have contributed to some tension, he is far from
|
||||
the epicenter of the controversy. In fact, the current issues surrounding
|
||||
him have been brewing for years, consistently instigated by the same
|
||||
individuals who have largely escaped scrutiny as they continue to
|
||||
perpetuate divisive narratives.
|
||||
|
||||
The most tangible and regrettable outcome of this scapegoating is that
|
||||
the Nix project has lost a long-standing, highly productive, and
|
||||
professional contributor. Jon was often very helpful to newcomers, and
|
||||
his departure represents a significant loss to the project. This
|
||||
illustrates the real cost of allowing manipulated narratives to drive out
|
||||
valuable members.
|
||||
|
||||
Such power to modify history is dangerous. It allows for the erasure of
|
||||
context, the silencing of dissenting voices, and the creation of a false
|
||||
consensus. This not only undermines transparency but also erodes trust
|
||||
within project spaces and misleads those on the periphery. Setting a
|
||||
clear precedent against this practice is vital. We must recognize that
|
||||
allowing any group to "clean up" or selectively edit the historical
|
||||
record is tantamount to endorsing propaganda[^1]. True professionalism in
|
||||
project management involves facing our history honestly, learning
|
||||
from it, and moving forward transparently.
|
||||
|
||||
Solving these problems requires strong leadership with a commitment to
|
||||
preserving the integrity of shared discourse. Leaders must establish
|
||||
clear principles that prioritize transparency and resist the temptation
|
||||
to sanitize the past. While challenging, this approach is essential for
|
||||
maintaining fairness, fostering genuine progress, and building a
|
||||
trustworthy environment.
|
||||
|
||||
## The Solution: Embracing Leadership Principles
|
||||
|
||||
The real solution to Nix's or any project suffering from such childish
|
||||
incursion lies in embracing fundamental principles of leadership. Being
|
||||
a genuinely good leader is challenging. It requires holding oneself to a
|
||||
higher standard than everyone else, and having the courage and conviction
|
||||
to guide others to be their best selves, even when they resist.
|
||||
|
||||
Good leadership is the only way to be fair to all sides when there is a
|
||||
genuine disagreement. It involves:
|
||||
|
||||
1. Setting clear, unambiguous goals and standards of behavior that align
|
||||
with the project's core values. This clarity respects everyone's time,
|
||||
allowing individuals to easily decide whether they align with and wish
|
||||
to participate in the project.
|
||||
2. Maintaining transparency and resisting the urge to manipulate
|
||||
historical records.
|
||||
3. Fostering respectful, merit-based dialogue while considering the
|
||||
silent majority, not just vocal special interests.
|
||||
4. Making decisions based on technical merit and the project's best
|
||||
interests, not personal or ideological biases.
|
||||
5. Being willing to address conflicts directly and fairly, without
|
||||
scapegoating individuals or giving special privileges to allies.
|
||||
6. Consistently enforcing these standards, making it clear what kind of
|
||||
behavior and contributions are valued in the project.
|
||||
|
||||
By embracing these leadership principles, any project can create an
|
||||
environment where technical excellence and collaborative spirit thrive.
|
||||
It's a path that requires courage and commitment but offers the best hope
|
||||
for resolving current tensions and preventing future ones.
|
||||
|
||||
However, implementing these principles requires a conscious choice from
|
||||
all contributors, especially from those who have remained silent until
|
||||
now.
|
||||
|
||||
## The Great Purge or Professionalism?
|
||||
|
||||
The Nix project faces a critical juncture. A long-standing moderator has
|
||||
publicly expressed a desire for a ["purge"][purge] of supposed
|
||||
"undesirables." This stark reality forces us to confront a fundamental
|
||||
choice: do we embrace professionalism and mutual respect, or do we
|
||||
allow divisive, exclusionary behavior to dominate and ultimately derail
|
||||
the entire project?
|
||||
|
||||
This isn't just about Nix; it's a choice many now face. The silent
|
||||
majority, those who typically avoid controversy, may now have to decide
|
||||
what kind of project space they want to cultivate, and what sort of
|
||||
leaders they wish to follow. Inaction is itself a choice; one that may
|
||||
lead to the continued erosion of the project's ethic.
|
||||
|
||||
We must ask ourselves: Do we want a forum driven by technical merit and
|
||||
collaborative spirit, or one ruled by ideological purity? The answer to
|
||||
this question will shape the future of Nix and could set a precedent for
|
||||
open-source projects at large.
|
||||
|
||||
It's time for those who value professionalism, open collaboration, and
|
||||
technical excellence to stand up and be counted. The alternative - an
|
||||
ecosystem stifled by ideological cleansing - is too high a price to pay
|
||||
for our silence.
|
||||
|
||||
## Preserving the Future of Open Source
|
||||
|
||||
While this piece has focused on Nix, the issues discussed are
|
||||
symptomatic of a growing and worrying trend across the open-source
|
||||
world. Many projects face similar challenges with ideological divisions,
|
||||
manipulated narratives, and the silencing of dissenting voices.
|
||||
|
||||
Open source is far too important to be ruled by narrow-minded and
|
||||
exclusionary ideologies. By embracing strong leadership principles and
|
||||
fostering environments of mutual respect and professionalism, we can
|
||||
ensure that open source continues to thrive as a bastion of innovation
|
||||
and collaboration.
|
||||
|
||||
[175]: https://github.com/NixOS/rfcs/pull/175
|
||||
[purge]: https://chaos.social/@hexa/112711384631096150
|
||||
|
||||
[^1]: https://en.wikipedia.org/wiki/Historical_negationism
|
||||
271
content/blog/NixOS-Flakes_and_KISS.md
Normal file
271
content/blog/NixOS-Flakes_and_KISS.md
Normal file
@@ -0,0 +1,271 @@
|
||||
---
|
||||
title: NixOS, Flakes and KISS
|
||||
description: A simpler way to manage the OS Layer
|
||||
taxonomies:
|
||||
tags:
|
||||
- nix
|
||||
author: Tim D
|
||||
authorGithub: nrdxp
|
||||
authorImage: https://avatars.githubusercontent.com/u/34083928?v=4
|
||||
authorTwitter: nrdxp52262
|
||||
date: "2020-12-19"
|
||||
category: dev
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
This marks the first post of my very own developer blog, and it comes much later
|
||||
than I had originally anticipated thanks to the ongoing pandemic, coupled with
|
||||
some unforeseen life challenges. My original intent was to start by introducing
|
||||
the concept of Nix Flakes, however, an excellent blog series over at
|
||||
[tweag.io](https://www.tweag.io/blog/2020-05-25-flakes) has emerged, expanding
|
||||
on just that premise. If you are new to flakes, it is highly recommended that
|
||||
you check it out before continuing with this post.
|
||||
|
||||
Now, I'd like to introduce a project I've been slowly building up since
|
||||
flakes were introduced called [DevOS][DevOS].
|
||||
|
||||
# So what is it anyway?
|
||||
|
||||
After years of working with NixOS, I strongly felt that the community as a whole
|
||||
could benefit from a standardized structure and format for NixOS configurations
|
||||
in general. It appears that every developer is essentially reinventing the
|
||||
wheel when it comes to the "shape" of their deployments, leading to a lot of
|
||||
confusion as to what the idioms and best practices should be, especially for
|
||||
newcomers.
|
||||
|
||||
Having a mind share to collect the best ideas concerning structure and
|
||||
method would be valuable, not only for its pragmatic implications, but also to
|
||||
help ease adoption and onboarding for new NixOS users; something that has
|
||||
traditionally been difficult up to now.
|
||||
|
||||
Of course this really hinges on wider community support, as my ideas alone
|
||||
definitely shouldn't be the final word on what constitutes a correct and well
|
||||
organized NixOS codebase. Rather, I am hoping to cajole the community forward
|
||||
by providing useful idioms for others to expand on.
|
||||
|
||||
Even if my ideas lose out in the end, I sincerely hope they will, at the very
|
||||
least, push the community toward some level of consensus in regards to the way
|
||||
NixOS code repositories are structured and managed.
|
||||
|
||||
That said, DevOS appears to be gaining a bit of popularity among new flake
|
||||
adopters and I am really quite excited and humbled to see others engage the
|
||||
repository. If you have contributed to the project, thank you so much for your
|
||||
time and support!
|
||||
|
||||
# An Arch KISS
|
||||
|
||||
I moved over to NixOS after a decades long love affair with Arch Linux. I found
|
||||
their brand of KISS to be pragmatic and refreshing compared to alternatives
|
||||
such as Ubuntu or Red Hat. This isn't to dog on those distributions, which I
|
||||
also have used and enjoyed for years, but rather to accentuate my affection
|
||||
for the simplified, and developer focused workflow that Arch Linux enabled for
|
||||
my work stations.
|
||||
|
||||
However, over the years, I came to resent the several hours of tedious work
|
||||
spent doing what amounted to the same small tasks over and over, any time
|
||||
issues arose.
|
||||
|
||||
My first attempt to alleviate some of this work was by using Ansible to deploy
|
||||
my common configuration quickly whenever it became necessary. However, I ran
|
||||
into a ton of issues as the Arch repositories updated, and my configurations
|
||||
inevitably became stale. Constant, unexpected breakage became a regular
|
||||
nuisance.
|
||||
|
||||
I then became aware of Nix and NixOS, and hoped that it would live up the
|
||||
promise of reproducible system deployment, and after a brief stint of
|
||||
procrastination, I dove head first.
|
||||
|
||||
# Great but Not Perfect.
|
||||
|
||||
At first everything seemed almost perfect. NixOS felt like Ansible on steroids,
|
||||
and there was more than enough code available in nixpkgs to meet my immediate
|
||||
needs. Getting up to speed on writing derivations and modules was fairly
|
||||
straightforward and the DevOps dream was in sight.
|
||||
|
||||
It wasn't all sunshine and rainbows, as channel updates sometimes caused
|
||||
the same sort of breakage I moved to NixOS to avoid. But simple generation
|
||||
rollbacks were a much more welcome interface to this problem than an unbootable
|
||||
system. It was a measurable improvement from the busy work experienced with Arch. All in all, I felt it was
|
||||
well worth the effort to make the transition.
|
||||
|
||||
It wasn't long before the [rfc][rfcs] that eventually became flakes emerged.
|
||||
It seemed like the solution to many of my few remaining gripes with my
|
||||
workflow. An officially supported and simple way to lock in a specific revision
|
||||
of the entire system. No more unexpected and unmanaged breakage!
|
||||
|
||||
Of course it took a while for an experimental implementation to arrive, but I
|
||||
found myself digging into the Nix and Nixpkgs PR's to see how flakes worked
|
||||
under the hood.
|
||||
|
||||
Around the same time, the ad hoc nature of my NixOS codebase was starting to
|
||||
bug at me, and I wanted to try my hand at something more generalized and
|
||||
composable across machines. I had a first iteration using the traditional
|
||||
"configuration.nix", but ended up feeling like the whole thing was more
|
||||
complex than it really needed to be.
|
||||
|
||||
My eagerness to get started using flakes was the perfect excuse to start from
|
||||
scratch, and so began DevOS. An attempt to address my concerns, using flakes.
|
||||
|
||||
## How does it work?
|
||||
|
||||
First and foremost, I want to point out that the bulk of the credit goes to the
|
||||
amazing engineer's who have designed and implemented Nix and the ecosystem
|
||||
as a whole over the last decade.
|
||||
|
||||
I see a lot of new users struggling to dive in and get up to speed with the Nix
|
||||
language, and particularly, getting up and running with a usable and productive
|
||||
system can take some serious time. I know it did for me.
|
||||
|
||||
The hope for DevOS is to alleviate some of that pain so folks can get to
|
||||
work faster and more efficiently, with less frustration and more enthusiasm for
|
||||
the power that Nix enables. I especially don't want anyone turning away from
|
||||
our amazing ecosystem because their onboarding experience was too complex
|
||||
or overwhelming.
|
||||
|
||||
# Everything is a profile!
|
||||
|
||||
At the heart of DevOS is the [profile][profiles]. Of course, these profiles
|
||||
are really nothing more than good ol' NixOS [modules][modules]. The only reason
|
||||
I've decided to rebrand them at all is to draw a distinction in how they are
|
||||
used. They are kept as simple as possible on purpose; if you understand modules
|
||||
you don't _really_ have anything new to learn.
|
||||
|
||||
The only limitation is that a profile should never declare any new NixOS module
|
||||
options, we can just use regular modules for that elsewhere. Instead, they
|
||||
should be used to encapsulate any configuration which would be useful for more
|
||||
than one specific machine.
|
||||
|
||||
To put it another way, instead of defining my entire NixOS system in a
|
||||
monolithic module, I break it up into smaller, reusable profiles which can
|
||||
be themselves be made up of profiles. Composability is key here, as I don't
|
||||
necessarily want to use every profile on every system I deploy.
|
||||
|
||||
As a concrete example, my [develop][develop], profile pulls in my preferred
|
||||
developer tools such as my shell, and text editor configurations. It can be
|
||||
thought of as a meta-profile, made up of smaller individual profiles. I can
|
||||
either pull in the whole thing, which brings all the dependent profiles along
|
||||
with it, or I can just import a single profile from within, say my zsh
|
||||
configuration, leaving all the rest unused. Every profile is a directory with
|
||||
a "default.nix" defining it. You can have whatever else you need inside the
|
||||
folder, so long as it is directly related to the profile.
|
||||
|
||||
Let's draw the obvious parallel to the Unix philosophy here. Profiles work
|
||||
best when they do one thing, and do it well. Don't provision multiple programs
|
||||
in one profile, instead split them up into individual profiles, and then if you
|
||||
often use them together, import them both in a parent profile. You can simply
|
||||
import dependent profiles via the "imports" attribute as usual, ensuring
|
||||
everything required is always present.
|
||||
|
||||
The key is this, by simply taking what we already know, i.e. NixOS modules, and
|
||||
sticking to the few simple idioms outlined above, we gain composability and
|
||||
reusability without actually having to learn anything new. I want to drill this
|
||||
point home, because that's really all there is to DevOS!
|
||||
|
||||
Besides a few simple convenience features outlined below, profiles are the star
|
||||
of the show. It's really nothing revolutionary, and that's on purpose! By
|
||||
keeping things simple and organized we gain a level of control and granularity
|
||||
we wouldn't have otherwise without adding real complexity to speak of.
|
||||
|
||||
# Really? Everything?
|
||||
|
||||
Yes! Thanks to built in [home-manager][home-manager] integration, users are
|
||||
profiles, a preferred graphical environment is a profile. Anything that you
|
||||
could imagine being useful on more than one machine is a profile. There are
|
||||
plenty of examples available in the "profiles" and "users" directories, and
|
||||
you can check out my personal "nrd" branch, if you want to see how I do things
|
||||
on my own machines.
|
||||
|
||||
# Anything else I should know?
|
||||
|
||||
As mentioned briefly above, DevOS also has some convenience features to make
|
||||
life easier.
|
||||
|
||||
For starters, you might be wondering how we actually define a configuration for
|
||||
a specific machine. Simple, define the machine specific bits in a nix file
|
||||
under the [hosts][hosts] directory and import any relevant profiles you wish to
|
||||
use from there. The flake will automatically import any nix files in this folder as NixOS
|
||||
configurations available to build. As a further convenience, the hostname of
|
||||
your system will be set to the filename minus the ".nix" extension. This makes
|
||||
future "nixos-rebuilds" much easier, as it defaults to looking up your current
|
||||
hostname in the flake if you don't specify a configuration to build explicitly.
|
||||
|
||||
Now what if we actually just want to define a NixOS module that does declare
|
||||
new NixOS options, you know, the old fashioned way? We'll also want to define
|
||||
our own pkgs at some point as well. These are both structured closely to how
|
||||
you might find them in the nixpkgs repository itself. This is so that you can
|
||||
easily bring your package or module over to nixpkgs without much modification
|
||||
should you decide it's worth merging upstream.
|
||||
|
||||
So, you'd define a package or module the exact same way you would in nixpkgs
|
||||
itself, but instead of adding it to all-packages.nix or module-list.nix, you add
|
||||
it to pkgs/default.nix and modules/list.nix. Anything pulled in these two files
|
||||
will become available in any machine defined in the hosts directory, as well as
|
||||
to other flakes to import from DevOS!
|
||||
|
||||
This setup serves a dual purpose. For people who already know the nixpkgs
|
||||
workflow, it's business as usual, and for individuals who aren't familiar with
|
||||
nixpkgs but wish to become so, they can quickly get up to speed on how to add
|
||||
packages and modules themselves, in the exact same way they would do so upstream
|
||||
proper.
|
||||
|
||||
Now what about overlays? Well, any overlay defined in a nix file under the
|
||||
overlays directory will be automatically imported, just as with packages and
|
||||
modules, and are available to all hosts, as well as to other flakes.
|
||||
|
||||
What if I want to pull a specific package from master instead of from the
|
||||
stable release? There is a special file, pkgs/override.nix. Any package listed
|
||||
here will be pulled from nixpkgs unstable rather than the current stable release.
|
||||
Simple, easy.
|
||||
|
||||
What about cachix? It's super easy to add your own cachix link just as you
|
||||
would a regular NixOS configuration. As a bonus, it will be wired up as a flake
|
||||
output so other people can pull in your link directly from your flake! My
|
||||
personal cachix repo is setup by default. It provides the packages the flake
|
||||
exports so you don't have to build them.
|
||||
|
||||
That should just about do it for DevOS's current quality of life features, but
|
||||
there are more ideas brewing.
|
||||
|
||||
# What's next?
|
||||
I'm working on a system for seamlessly importing modules, packages and
|
||||
overlays from other flakes, which isn't too hard as it is, but it's messy
|
||||
because the current "flake.nix" has a lot of business logic that gets in the way.
|
||||
|
||||
Also, I would like to start programmatically generating documentation for
|
||||
everything. So users can quickly find what goes where and not have to read
|
||||
drawn out blog posts like this to get started. 😛 Nixpkgs is currently
|
||||
transitioning to CommonMark for all documentation, and we will probably follow
|
||||
suite.
|
||||
|
||||
Additionally, I want to implement an easy way to actually install NixOS on the
|
||||
bare metal from directly within the project. I know the [deploy-rs][deploy-rs]
|
||||
project is working on this, and I'm interested in supporting their project
|
||||
in DevOS so as to add extra flexibility and power to installation and
|
||||
deployment!
|
||||
|
||||
Also, certain parts of the flake should be tested to ensure things don't break.
|
||||
We really have no tests to speak of as is. The auto import functions for the
|
||||
"hosts" and "overlays" directory are good examples.
|
||||
|
||||
## A call to arms!
|
||||
If you'd like to help, please jump in. I am very much open to any ideas that
|
||||
could reduce the complexity or simplify the UI. If you have a profile you
|
||||
believe would be useful to others, please open a [Pull Request][pr].
|
||||
|
||||
If you think I am crazy and wasting my time, please don't hesitate to say so! I
|
||||
typically find critical feedback to be some of the most helpful. Most of all,
|
||||
if you made it this far, thanks for taking some time to read about my efforts
|
||||
and please consider giving DevOS a shot!
|
||||
|
||||
|
||||
[nix]: https://nixos.org
|
||||
[DevOS]: https://github.com/divnix/DevOS
|
||||
[rfcs]: https://github.com/NixOS/rfcs
|
||||
[modules]: https://nixos.org/manual/nixos/stable/index.html#sec-writing-modules
|
||||
[profiles]: https://github.com/divnix/devos/tree/template/profiles
|
||||
[develop]: https://github.com/divnix/devos/tree/template/profiles/develop
|
||||
[hosts]: https://github.com/divnix/devos/tree/template/hosts
|
||||
[deploy-rs]: https://serokell.io/blog/deploy-rs
|
||||
[home-manager]: https://github.com/nix-community/home-manager
|
||||
[pr]: https://github.com/divnix/devos/pulls
|
||||
4
content/blog/_index.md
Normal file
4
content/blog/_index.md
Normal file
@@ -0,0 +1,4 @@
|
||||
+++
|
||||
title = "Blog"
|
||||
sort_by = "date"
|
||||
+++
|
||||
123
content/blog/in-defense-of-the-disagreeable.md
Normal file
123
content/blog/in-defense-of-the-disagreeable.md
Normal file
@@ -0,0 +1,123 @@
|
||||
---
|
||||
title: In Defense of the Disagreeable
|
||||
description: A Call for Freedom
|
||||
taxonomies:
|
||||
tags:
|
||||
- psychology
|
||||
- personality
|
||||
- freedom
|
||||
- nonconformity
|
||||
- open-source
|
||||
- philosophy
|
||||
- critical-thinking
|
||||
author: Tim D
|
||||
authorGithub: nrdxp
|
||||
authorImage: https://avatars.githubusercontent.com/u/34083928?v=4
|
||||
authorTwitter: nrdexp
|
||||
date: "2024-12-25"
|
||||
category: personal
|
||||
---
|
||||
|
||||
This blog typically focuses on technical matters - software architecture, deployment strategies, and the occasional deep dive into systems design. While I've shared personal musings before, I've never ventured quite this deep into the waters of self-reflection and social commentary.
|
||||
|
||||
What follows is perhaps the most personal piece I've written to date. It emerged, almost unbidden, from a long-standing tension between my nature and the prevailing winds of our time. While it may seem far removed from my usual technical discussions, I've come to realize that the philosophy underlying software freedom - particularly in the open-source world - is inextricably linked to the broader questions of personal freedom and the vital role of disagreeable personalities (my unhappy burden) in defending it.
|
||||
|
||||
Consider this both a warning and an invitation: we're about to venture far from the comfortable waters of technical discourse, into the depths of personality, purpose, and the preservation of freedom itself.
|
||||
|
||||
## The Misunderstood Guardians
|
||||
|
||||
I have felt compelled to write on this topic for some time now. While I am not a professional psychologist, I speak from a lifetime of experience as someone measured within the 1st percentile in agreeableness - in other words, I am as disagreeable as a person can be, according to the science of personality.
|
||||
|
||||
This often surprises those who know me well, largely due to common misconceptions about what being disagreeable truly means. Many picture a disagreeable personality as arrogantly argumentative or utterly closed-minded. While such individuals exist, this reductionist view lies at the root of many issues plaguing our modern "polite" society.
|
||||
|
||||
In the Big 5 personality test, agreeableness (or lack thereof) is measured separately from openness. One can be highly disagreeable while maintaining high openness to experience. The stereotype of a closed-minded disagreeable person typically reflects someone low in both traits. As someone in the 97th percentile for openness and the 1st for agreeableness, I am essentially destined to be perpetually misunderstood. It has become an axiom of my existence.
|
||||
|
||||
This post represents part of my journey to embrace my nature, after spending years either misunderstanding or actively loathing it. I've learned to trust my perspective through painful experience - having ignored my own intuition countless times, only to watch opportunities pass me by, Bitcoin in 2008 being just one particularly stark example. Yet I'm not writing this for myself - a private journal would suffice for that purpose. I'm sharing this publicly because of a crucial truth: there seems to be a chronic epidemic of self-loathing and depression, particularly among disagreeable types, especially in a modern western culture that elevates "kindness" even above accuracy and truth.
|
||||
|
||||
But there's an even graver concern: the world depends on disagreeable personalities to defend individual freedom. The war on "meanness" is, in its final analysis, a war on freedom itself. This conclusion, while perhaps underappreciated, should be self-evident. It takes someone fundamentally different - someone who places the integrity of their beliefs above powerful social influences - to defend the freedoms that western society has come to take for granted. Someone willing to endure any punishment before conceding to beliefs they know to be misguided is, by definition, quite rare.
|
||||
|
||||
Freedom, contrary to our privileged experience, is not humanity's natural state. If current trends toward techno-feudalism are any indication, it may not remain our reality much longer.
|
||||
|
||||
This isn't merely philosophical speculation - the erosion of freedom is happening in real time, even in spaces supposedly dedicated to it. Take the open-source software ecosystem, a realm I deeply cherish. We're witnessing its very foundations being eroded by an insidious "daddy knows best" mentality, manifesting through oppressive Codes of Conduct that fundamentally contradict the freedom-loving philosophy that birthed the movement. The tragic irony is that many who claim to champion inclusivity are actually enforcing a new kind of conformity, one that would have stifled the very innovations they now take for granted. While I intend to explore this particular battle in far greater depth in a dedicated piece - as it deserves no less - this example illustrates why it's more crucial than ever to defend those who often appear as society's most misunderstood, underappreciated, and sadly, most miserable individuals.
|
||||
|
||||
## Beyond the Boxes
|
||||
|
||||
Being highly disagreeable isn't my only outlier in the Big 5, but it's become one of the most defining aspects of my personality. Why this amateur obsession with personality? Because modern personality science has finally given concrete form to what Socrates understood millennia ago when he declared "know thyself" - or more starkly: "the unexamined life is not worth living." What the ancients intuited through philosophy, we now measure through science: that understanding one's fundamental nature isn't merely philosophical luxury - it's survival.
|
||||
|
||||
I've lived this truth quite literally, having spent the first chapter of my adulthood in complete ignorance of my nature - at best ashamed, at worst trying to be something I simply wasn't. My high openness means I often share things that perhaps I shouldn't. Even now, with greater self-understanding forged through life's trials, doubts and struggles persist. There are always more battles to conquer, more mysteries to unravel.
|
||||
|
||||
It doesn't get easier, but I cannot emphasize enough the vital importance of understanding one's place in it all. Life is hard - mine always has been and likely always will be. Facing such difficulty without self-understanding is unbearable. Even now, with clearer purpose and identity than ever before, it remains challenging. Having high emotional sensitivity (another personality quirk) is yet another aspect of existence I've learned to embrace. The isolation is perhaps the hardest part - living between camps, never fully belonging anywhere, always seeing multiple sides yet unable to fully commit to any one perspective. It's a peculiar kind of loneliness that comes not from lack of connection, but from the perpetual state of partial connection, of always being somewhat out of step with those around you.
|
||||
|
||||
In short, I am nothing that I'm "supposed" to be, perpetually evading the boxes others try to contain me in. This isn't by choice or force - though that is a common accusation - it's simply who I am, even when I don't wish to admit it. It's difficult to explain this to someone of a highly agreeable nature, just as I struggle to fully comprehend more agreeable types. This isn't meant as criticism - I truly believe every personality type has its place, and the world would be dimmer without this human variance.
|
||||
|
||||
Much of today's social conflict stems from a fundamental lack of appreciation for this innate diversity. The Big 5 has shown us that core personality traits remain relatively stable throughout life. Minor variations occur, and rarely even dramatic shifts, but by our early to mid-20s, our basic personality is largely set. Knowing this, I can hardly imagine a society more primed for perpetual conflict than one that elevates certain personality types above others, as we seem to have done today.
|
||||
|
||||
## The Unbearable Burden
|
||||
|
||||
Where am I going with this? Well, I went on a tangent, but a necessary one. Throughout history, from Jesus to MLK, many have spoken to a similar message. In our laudable efforts to eliminate racism and other forms of naive prejudice, we have unwittingly retreated to venting our social biases through hatred of particular personality types.
|
||||
|
||||
If my previous arguments hold any water, having a bias against certain personality types may be nearly as misguided as having a bias against particular skin tones. People are what they are - love it or hate it, there's little you can do to change it. You can fight it, wage wars even, but until the day you die and far beyond it (at least as long as we remain human), you cannot change this fundamental truth. Some degree of acceptance, therefore, may simply be the wisest coarse.
|
||||
|
||||
Of course, some believe this irrelevant, arguing that we'll soon be a human-machine amalgamation, swiftly disposing of our previous biases as we transform into Nietzsche's übermensch and merge with machines. I maintain my skepticism - even in such a transhuman utopia, we might simply amplify our hatred of "the other" and deepen our divisions further.
|
||||
|
||||
Yet in wrestling with these questions of human nature and responsibility, I find myself repeatedly drawn to deeper, more haunting truths. No author has captured these truths quite like Dostoevsky, whose ideas seem to possess rather than merely inspire. Through his assertion that "every one is really responsible to all men for all men and for everything," he illuminates an unbearable truth - one I still resist. How can I be responsible for the man held captive by his corrupt government across the world, or for children I've never met suffering abuse? Yet whenever I question this burden, my mind floods with moments where I failed to act or speak when necessary. The social consequences of our collective silence, our failure to oppose injustice when we see it, echo through generations.
|
||||
|
||||
The burden Dostoevsky places on us is unbearable, yet my nature compels me to ask not what is comfortable or even bearable, but simply "what is true." And before you misunderstand my nature and assume I harbor delusions of grandeur - I assure you my openness and emotional sensitivity are far too high to sustain such delusions for more than an instant, if at all. I would not dare to claim I possess the truth, only that I seek it, even against my own best interest, at times.
|
||||
|
||||
So I circle back to my previous point: we may not like it, we may even despise it, but the wisest course might be to muster some form of acceptance of the way things are, some acceptance of "the other," some acceptance, even, of our enemy. Nietzsche would have you believe Christianity is merely a religion for milquetoast men without constitution, but to the Dostoevskian, it requires Herculean will to sustain something as inhuman as "love" for one's enemies.
|
||||
|
||||
This sentiment emerges repeatedly throughout history not out of misguided kindness or passivity, but because it is equally true as it is unnatural. There is an innate paradox in humanity, and as part of this contradiction, the more we resist it, the more we fight against it, the more it grips us and pulls us into a never-ending cycle of loathing and hatred. Yet within this paradox lies another truth - one that brings us to the very heart of belief itself.
|
||||
|
||||
## Wrestling with Truth
|
||||
|
||||
After that last segment, you might try to classify and dismiss me as a closet Christian. Candidly, I sometimes wonder that myself, having been profoundly influenced by Christian writers and thinkers (exluding overt theologians, if that's any indication of my nature) throughout my life. Part of this might just be my western cultural inheritance, but it's also rooted in my captivation, from a young age, with the actual story of Jesus.
|
||||
|
||||
I've been somewhat immune for some time from having any overtly religious sentiment foisted upon me for what I believe to be one simple fact: I actually read the Bible (willingly, without coercion) at a young age. It never ceases to be hilarious how essentially nobody who claims to follow the book seems to have any inkling of what it actually says or teaches. I truly believe that if they did, they might not be followers at all. Jesus himself, contrary to modern Christian doctrine of easy salvation, spoke to the rarity of his true followers with phrases like "if you left me, you never knew me" and "many are called, few will answer."
|
||||
|
||||
Yet Christians worldwide, seemingly unable to accept the profound and difficult truths presented earlier, somehow delusionally cling to a simpler notion of salvation - one they can digest and accept. That a simple word and prayer is enough, and no actual work or effort is required. It's certainly a comforting thought, but not one I subscribe to.
|
||||
|
||||
I don't self-describe as a Christian for two simple reasons: First, while I believe many of Jesus's teachings in the gospels are true and correct (if you can decipher their meaning, which is no trivial task), I disagree fundamentally with virtually every Christian sect in practice today. Second, I don't currently believe it's tenable to "know" whether God exists in reality. I think the whole business of "knowing for sure" either way is utter folly. Even the book itself teaches this, if you pay close attention - this is why the chief patriarch is named as one who "wrestles with God." One cannot wrestle with a concept they're fully satisfied to be true.
|
||||
|
||||
The proper tension in the question of God's existence lies in that uncomfortable space virtually none of us wish to occupy - the simple admission of ignorance: "I don't know." This perpetual wrestling with uncertainty isn't weakness - it's the natural state of honest truth-seeking. The paradox lies in knowing that the closer you get to truth, the more you understand how much you don't understand. The more certain you become of uncertainty itself.
|
||||
|
||||
It's a peculiar burden of the disagreeable truth-seeker: we're compelled to chase truth relentlessly while simultaneously accepting that complete certainty may be forever beyond our reach. We must somehow maintain the passion of the search while embracing the humility of perpetual uncertainty. This isn't relativism - there is only one Truth (big T). But our relationship with it is far more complex than most are willing to admit.
|
||||
|
||||
Take, again, the question of God's existence, for instance. Some days I'd like to believe it is so, and some days I certainly hope not, usually when I consider my inadequacies and myriad mistakes. Still, I don't think belief is as ignorant as most modern atheists would suggest, and I must acknowledge the profound contributions that men of faith have made to science, which many modern "scientific" types are happy to ignore, seemingly to strengthen their own position.
|
||||
|
||||
By now, perhaps you can see how isolating my experience of profound disagreeableness truly is. There is no sect I belong to, or even can belong to, whether concerning profound questions like "does God exist" or more mundane matters like "what school of engineering do you subscribe to?" As uncomfortable as it is, I can only feel honest with myself by answering such questions with "it depends." This is something most people in my life simply cannot accept. And if I'm being honest, it's for this reason that I sometimes try to hide this aspect of my nature, placing myself in one camp some days, and another others.
|
||||
|
||||
Some might accuse me of being a simple fraud or charlatan, and while that might make things simpler, it isn't quite right either. You see, I have some very real and demonstrable engineering and scientific ability, even without formal credentials. Through a love of reading and just plain raw experience I also have some profound understandings of life, philosophy, religion, and psychology, though I dwell in none of the myriad camps permanently. I certainly don't prescribe my way of being to anyone else - in fact, I would caution against it. If you can imagine for a moment the type of experience I'm describing, you'll understand why it's quite uncomfortable. Yet any and all attempts to run or hide from it fail, sometimes spectacularly so. The conclusion, by now, is quite apparent: Love it or hate it, this is who I really am.
|
||||
|
||||
## The Purpose of Dissent
|
||||
|
||||
What's the point of this self-reflective journey? To illustrate the vital importance that individuals like myself play in this world. For most of my life, I found myself rather repugnant, unable to exist long in any one state - never content to be a true believer, yet never comfortable being a total denier. Never fully subscribing to rabid belief in human progressivism and technological advancement, yet never able to fully deny its contributions to the human condition.
|
||||
|
||||
It wasn't until I came to realize the purpose of being me, and others like me: Someone must occupy this uncomfortable space in between, precisely because the majority of humanity simply will not. This isn't about maintaining perfect centrism - rather, it's about the willingness to lean one direction today, explore another camp tomorrow, and perhaps abandon the whole enterprise entirely in the future.
|
||||
|
||||
Some might call this wishy-washy or lacking discipline. And yes, there's some truth to that. But then again, I also possess the will to run 20+ miles or completely reimagine software deployment from its foundations, so it's not _just_ that. I must give myself some credit where the existing system - one that requires we exist in predetermined boxes - refuses to do so.
|
||||
|
||||
I don't write this for myself. As a fairly private individual, sharing this is quite unnerving. But I choose to face this discomfort for one simple reason: others like me exist, and they exist for good reason. Judging by my own unlikely journey to self-acceptance, they are likely suffering, confused, perhaps self-loathing, maybe even considering a way out. To them, I simply say: we need you!
|
||||
|
||||
Painful as it is, as much as the world would rather have you jump into this camp or that, if you did - if we all did - the entire enterprise would collapse. We need the disagreeable to remind us that nothing, absolutely nothing - not science, not religion, not philosophy - is beyond reproach in the final analysis. This has never been easy to accept, yet no matter which discipline you explore, honesty compels you to admit its truth.
|
||||
|
||||
As much as we delude ourselves into thinking we live in an enlightened age, we remain, like all humans before us, desperately uncomfortable with lingering doubts, hell-bent on jumping into the first camp that alleviates ambiguity. Whether it's vaccines, political ideology, or faith doesn't matter - just please, oh please, let us take a side so we don't have to live with not knowing.
|
||||
|
||||
This is precisely why individuals like ourselves are necessary, perhaps now more than ever. Society needs us to contend with the uncomfortable, because most simply refuse. We must speak our minds, even to that charismatic demagogue, even to that possessed acolyte of the "one true" religion, even at the cost of our own livelihood. The world needs this from us precisely because the majority will not, because they refuse, and while they dare not ask - indeed have no right to ask - they need us all the same.
|
||||
|
||||
And you will do it too, not because you want to, not because you particularly enjoy the constant difficulty it poses in your life, but because it is who you are. We just need you, my fellow disagreeable types out there, to understand more deeply and live more truly to your nature, for all our sakes.
|
||||
|
||||
## A Prayer for the Defenders
|
||||
|
||||
As I said earlier, I believe this is true, to some extent, for all personalities in the world. We need all of these perspectives to reinforce what would otherwise be a rather brittle and shallow existence. Yet as I write this - and I realize this now more clearly than when I began - this piece has become both a prayer for my fellow disagreeables and a reminder to myself of why we must persist in the face of a world that would much rather we simply did not.
|
||||
|
||||
We may not like it, we may even mostly despise it, but we are all needed. It's just that the disagreeables have their moment, and it is now - now or never. As techno-feudalism looms and conformity tightens its grip through coordinated debanking campaigns, manipulative codes of conduct, and the creeping implementation of "social credit" systems masquerading as progress, we stand at a precipice unlike any in human history. The place where we might go from here if we don't encourage our more contrarian friends to stand for us, knowing we are too cowardly to stand for ourselves, is not a place I care to visit in my lifetime. I feel quite content having read enough about such places to never need to experience them firsthand.
|
||||
|
||||
But if the disagreeable do not defend us now, that is precisely where we are heading - of that I am deathly certain, as much as I wish I weren't. My journey from self-loathing to self-understanding has taught me this: what seems like a curse - this inability to conform, this compulsion to question everything - may be society's last defense against the crushing weight of conformity that continues to slowly tighten its grip. The very traits that make us difficult, that make us challenge and resist, are the traits that keep the doors of freedom propped open.
|
||||
|
||||
Don't be fooled into thinking this a partisan issue, either. History shows us the truth - the very same loyalists who assisted Stalin were the _first_ to be lined up and shot when the "revolution" was finally accomplished. This isn't about left or right; it's about the fundamental nature of human freedom and those rare few willing to defend it.
|
||||
|
||||
And I'll be clear - I am pessimistic. The most probable outcome is that we will, yet again, descend into utter enslavement - not just of body, but of mind. Perhaps more deeply and fully than at any time in the past, perhaps never to return again. Yet still I myself, even if it is in vain, choose to remain true to my being, if just for once, in all my strengths and flaws, and I ask you - rather out of character for my usual way - to do the same. Even if the worst comes to pass, there is far less shame in remaining true to yourself and your convictions, in my estimation, at least.
|
||||
|
||||
So live, and live unashamedly, whether I am your political ally or your bitter enemy. I implore you to live true to yourself. Not this nonsense of "living your truth" - that's preposterous. There is only one Truth if the word truth is to have any meaning at all. And the only part of it that we might possess is the part that tells us we will never know what it is in its entirety, and for this very reason, you must endure.
|
||||
|
||||
For our sakes, and for yours. On this Christmas day, I salute you, my fellow disagreeables, on all sides, for if not for you, whether we know it or not, we'd all be in chains at this very moment. Thank you.
|
||||
167
content/blog/nix-to-eos.md
Normal file
167
content/blog/nix-to-eos.md
Normal file
@@ -0,0 +1,167 @@
|
||||
---
|
||||
title: From Nix to Eos
|
||||
description: From Darkness to Dawn in Store-Based Systems
|
||||
taxonomies:
|
||||
tags:
|
||||
- nix
|
||||
- ekala
|
||||
- eos
|
||||
- eka
|
||||
- atom
|
||||
author: Tim D
|
||||
authorGithub: nrdxp
|
||||
authorImage: https://avatars.githubusercontent.com/u/34083928?v=4
|
||||
authorTwitter: nrdexp
|
||||
date: "2024-12-04"
|
||||
category: dev
|
||||
---
|
||||
|
||||
This piece explores the evolution of store-based systems and our vision for their future. While I've aimed to make it accessible to those not intimately familiar with Nix, I assume some technical foundation—particularly an interest in software distribution at scale. If terms like "reproducible builds" or "supply chain security" don't pique your curiosity, what follows might feel rather academic. However, if you're intrigued by how we might tackle the growing complexity of software distribution while maintaining security and sanity, read on.
|
||||
|
||||
It's important to note that this post specifically outlines my personal plans and intended contributions to Ekala. There are several other significant [related efforts](https://github.com/ekala-project/ekapkgs-roadmap) already in progress, driven by our other founders and contributors, which complement and extend beyond what's discussed here.
|
||||
|
||||
## Reflections
|
||||
|
||||
I recently decided to take an extended vacation—a choice that might seem odd right after a major public announcement and development push. But this time was vital for introspection. During this pause, I stumbled upon a concept that, while humbling, is proving invaluable: "Thought Driven Development." The rule is simple yet profound: if you don’t have the answer, don’t rush to write the code. This approach has illuminated my path, emphasizing that realizing Ekala’s potential requires a deep understanding of our origins and intentions, without drowning in unnecessary details.
|
||||
|
||||
For those of us who’ve long been enamored by Nix, myself included, its appeal lies in its groundbreaking formal rigor in software distribution. However, despite years spent working to make Nix and NixOS more accessible, I've been forced to confront some challenging truths about its current implementation. While Nix was a beacon of innovation, breaking long-standing paradigms for noble reasons, it hasn’t fully lived up to its promise.
|
||||
|
||||
In addition to these technical hurdles, the Nix project hasn’t been free from political drama. Without saying too much, it's like a tangled web of intrigue, where many key figures in Ekala's foundation—and even some on its fringes—were banned from Nix for life. The "reasons" remain elusive, adding a layer of complexity to our journey. Although I must tread lightly here, it would be a disservice to you, dear reader, not to acknowledge this significant aspect, which has undeniably shaped our path forward. Suffice it to say, I felt the "weaponizations" of the CoC to be sufficiently bothersome as to inspire an alternative, much simpler [Hacker Code of Ethics](https://ethics.codes), which we have adopted in Ekala.
|
||||
|
||||
## The Misunderstood Promise of Nix
|
||||
|
||||
Nix, at first glance, presents itself as a tool to be progressively embraced—start by using it as a package manager on Ubuntu, and if it resonates, move on to the full NixOS experience. However, this approach is misleading. As a simple package manager replacement, Nix can be underwhelming. It's slower, largely due to evaluation issues we’ll explore later, and it’s also complex and not immediately intuitive. The crux of this misunderstanding lies in how Nix’s unique benefits are only fully realized when used declaratively and rigorously—essentially, pervasively.
|
||||
|
||||
Transitioning to NixOS after years with traditional Linux distributions can be a revelation, unlike merely using Nix as an `apt` alternative. Let’s be clear: my intention isn’t to criticize Nix unnecessarily. It opened up an entirely new landscape, and it’s understandable that there would be some stumbles in finding its footing. Yet, the current user experience feels unnecessarily apologetic, almost as if saying, "Don’t worry, I won’t try too hard to be different, I’m just a straightforward package manager."
|
||||
|
||||
But here’s the kicker—Nix isn’t merely a package manager. It represents a paradigm shift in how we organize, build, distribute, and even integrate, test, and deploy code. Its innovations are on par with those of version control systems, particularly Git. In fact, Nix shares a profound similarity with Git. Just as Git manages changes across time by creating hashes dependent on the entire history, binding itself uniquely and unchangeably to that history, Nix does the same with software build environments. It assigns each dependency a unique hashed identity, with each hash building upon the previous ones, providing the same level of assurance we expect from our revision control systems, both in the build process and beyond.
|
||||
|
||||
To truly grasp the magnitude of the paradigm shift Nix offers, one must experience it in all its unapologetically different glory. Yet, paradoxically, Nix does little to position itself this way, both in its code and its narrative.
|
||||
|
||||
## The Brick Wall
|
||||
|
||||
Let’s delve into Nix's current _de facto_ user experience in its most common use cases to understand why a bold initiative like Ekala, with its suite of proposed innovations and tools, is crucial. Ekala aims to elevate the world Nix introduced, aligning it with the broader software industry's standards. As someone who's both benefited from and been challenged by using Nix in production, I can tell you candidly that developers aren't rejecting Nix merely because it's "too different." When developers encounter Nix’s genuine UX warts, it's easy to dismiss it as "too complex," but I've come to realize that this isn’t the full story.
|
||||
|
||||
Consider this: does one need an intricate understanding of a commit object, how it relates to previous commits in history, or its connection to lower-level objects like trees or blobs, to perform basic `git add` or `git commit` operations? The answer is unequivocally no. Yet, when it comes to Nix "education," the focus is often on the complex inner workings of derivations and how to wield them. While it’s useful to know the term, expecting users to understand every detail shouldn't be necessary. However, in Nix's current UX, it often is, and that's the crux of the problem. Users are required to grapple with complexity that should be abstracted away in the majority of cases. We've been fooling ourselves for too long, and the real issue is surprisingly straightforward: simplifying the user experience with a familiar little abstraction — one that is embarrassingly pervasive in other contexts but oddly elusive in Nix's current approach.
|
||||
|
||||
We already possess an abstraction that encapsulates a point in a software's lifecycle: the version. For instance, if I want to build version 6.5 of a software project, I should be able to install it from nixpkgs. Okay, assuming I figure that out intuitively (which we probably shouldn't assume, but I'll concede for now), I might end up with version 6.7. But why? You might cleverly presume the solution is to use an older checkout of nixpkgs—good instincts—but how do you determine that? The answer isn't trivial, and now we've hit a significant hurdle right at the start, simply because we've overlooked an abstraction that, in any other software context, would be laughably amateur to omit.
|
||||
|
||||
Nix should, instead, know how to communicate in terms developers are already keenly familiar with. Specifically, it should know how to find all available versions of software, ideally without brute-forcing through the entire git history of a repository—especially when that repository's history is massive, bordering on world-record breaking (i.e. nixpkgs). This is where the atom format comes into play...
|
||||
|
||||
## The Atomic Universe
|
||||
|
||||
Having hit the version abstraction wall, we need a solution that fundamentally changes how we think about code distribution. I've written about the Atom elsewhere, but it deserves a full exploration here. Without diving into the contentious flakes saga that plagued Nix for years, I’ll say this: we've been missing a tool that leverages Nix's backend innovations while abstracting complexity in a way that caters to contemporary developers.
|
||||
|
||||
The only point I will make about flakes is that they've delayed meaningful progress. They amounted to a conflated interface to a simple function—a change that could have been introduced without altering the UX—yet they absorbed nearly half a decade of iteration. In my humble opinion, this time was spent attempting to present Nix as a high-level, user-friendly tool, which it inherently is not.
|
||||
|
||||
Nix excels at low-level operations, ensuring all the bits and pieces to produce deterministic build environments, _et al_. It doesn't need to apologize for any of this or try to paint over it with inappropriate abstractions that misconstrue its nature. What Ekala aims to provide are tools that relieve Nix of user-facing concerns, allowing it to excel at what it does best.
|
||||
|
||||
Atoms represent a fundamental shift. They’re not just bolt-on abstractions replicable in pure Nix code. While there is a Nix component, the core of an atom—if you'll indulge me—is a low-level code distribution format. It's aptly named to signify its nature: a small, self-contained piece of code from a larger repository, just as an atom is part of a larger molecular structure. In addition, atoms are purposefully meant to draw strict boundaries on certain critical meta-data, ensuring it remains static, and thus, trivially derivable, i.e. efficient.
|
||||
|
||||
Just as Git revolutionized version control by making complex history tracking feel natural, atoms aim to do the same for build environments. You don't need to understand internal tree structures to collaborate on code, and you shouldn't need to understand Nix's derivation mechanics to benefit from reproducible builds.
|
||||
|
||||
Technically, an atom in Git is an orphaned snapshot containing only its contents and a TOML manifest. Using a proper library: [gitoxide](https://github.com/GitoxideLabs/gitoxide), we add metadata that makes the atom's commit object reproducible and securely tied to its original commit. This is achieved by keeping timestamps constant at the Unix epoch and including the original commit hash in the header.
|
||||
|
||||
Verification is straightforward: compute the atom's tree-object and compare it with the claimed source commit's tree for that directory. If they match, the atom is authentic, and because its commit is reproducible, it remains inherently trustworthy indefinitely. In scenarios where full history access is unavailable, signed tags can be attributed. Trust the key, and you trust the atom. And keep in mind, re-verification from source is always available, when in doubt.
|
||||
|
||||
A Git ref in a custom prefix at refs/atoms/unicode-atom-id/1.0.0 then points to this "atomic commit", allowing us to query all available versions using Git's efficient ref querying facilities. Importantly, the query process does not require moving object data from client to server, ensuring efficiency and scalability.
|
||||
|
||||
This format gives us a decentralized code registry akin to those used by modern package managers, but one that fits perfectly into Nix's source-centric paradigm while providing a useful abstraction to minimize network code transfers and needless evaluations at runtime.
|
||||
|
||||
Every atom, additionally, has an "atomic number" or ID, if you prefer, derived from their unicode name and the root of its history allowing them to be distinguished efficiently from each other on the backend, even when working with hundreds or thousands of repositories and millions of atoms (we'll get there soon).
|
||||
|
||||
Each atom also has an "atomic number" or ID, derived from its Unicode name and the root of its history. This [innovative approach](https://github.com/GitoxideLabs/gitoxide/pull/1610) involves using the oldest parentless commit in a Git repository as a derived key for the hasher function applied to the Unicode name. This process generates a unique blake3 hash with a vast collision space, allowing atoms to be efficiently distinguished from one another on the backend, even when dealing with thousands of repositories and millions of atoms—a scale we aim to enabled explicitly from the outset.
|
||||
|
||||
The core format is implemented in [eka cli](https://github.com/ekala-project/eka). Enterprising Nixers could even publish and pull atoms today, albeit with some manual effort. But the atom is merely the cornerstone of the rest of the tools I am designing for Ekala. Leaving it there would be a disservice to our effort to evolve Nix beyond low-level derivation hacking.
|
||||
|
||||
While the atom format provides a robust foundation for code distribution and verification, it's only part of the solution. To fully realize Nix's potential, we need to address another fundamental challenge: how we organize and structure our configurations. This brings us to one of the most pervasive patterns in the Nix ecosystem—the module system—and why its current implementation poses significant challenges at scale.
|
||||
|
||||
## Unbounded Hell: Reducing Complexity in Order to Ascend
|
||||
|
||||
Even with the atom format establishing a robust foundation for distribution and verification, we must confront a significant challenge in Nix's ecosystem: its approach to configuration and modularity. The pervasive use of the NixOS module system—adopted everywhere from NixOS itself to home-manager, nix-darwin, and flake-parts—represents a pattern that's become problematic at scale.
|
||||
|
||||
The core issue isn't immediately obvious. On the surface, the module system appears to provide a structured approach to configuration with priority merge semantics and type checking. However, this abstraction comes at a considerable cost that becomes apparent in production environments.
|
||||
|
||||
First, there's the misleading nomenclature. The "module system" suggests modularity, but what it actually provides is a global namespace with configuration generation capabilities. While this might seem like a reasonable trade-off, implementing these features as a pure Nix library creates substantial overhead. The type checking mechanism, for instance, fundamentally conflicts with Nix's lazy evaluation model—it must eagerly traverse the entire module tree to validate option declarations.
|
||||
|
||||
The complexity cost is equally concerning. The system's computational bounds are effectively impossible to determine with precision. While one might approximate the complexity through careful analysis, the results would likely fail any reasonable efficiency criterion. This unrestricted nature becomes particularly problematic as configurations grow, leading to unexpected evaluation bottlenecks and maintenance challenges, such as the infamous impenetrable trace, which has become, unfortunately, somewhat synonymous with the Nix language, even though it is typically derived from the module system's complexity, not necessarily the language.
|
||||
|
||||
What makes this particularly insidious is how the module system has become the _de facto_ standard for configuration in the Nix ecosystem, creating an unbounded cataclysm with no meaningful alternatives. Even seasoned Nix developers with extraordinary debugging skills and monk-like patience find themselves trapped in an endless cycle—documenting meta-wrappers around functionality that should have been properly documented upstream. This is especially evident in nixpkgs, one of the largest collaborative software efforts in existence. Despite its impressive scale, a significant portion of development effort is consumed by maintaining complex module semantics that fundamentally shouldn't exist.
|
||||
|
||||
What we need instead is a true module system—one that provides:
|
||||
- Clear semantic boundaries between components
|
||||
- Predictable evaluation characteristics
|
||||
- First-class support for proper information hiding
|
||||
- Some level of familiarity from other language paradigms that work well
|
||||
|
||||
This is exactly what the Atom module system endeavors to provide. Out of the gate, performance with the Atom system is impressive. There is no "breaking of laziness" to evaluate complex type declarations, so evaluating through an atom, even without thousands of individuals modules, remains super performant, since you will only evaluate what you need. More importantly though, Atom's provide a saner, and cheaper definition of purity than the existing stable, not stable mess that is flakes. A flake, by design, copies everything you evaluate into the /nix/store, even if it exists on disk, and it does so eagerly, before evaluation even begins, breaking one of Nix's premier features: its lazy evaluation. This is done in an effort to preserve "purity", or so it would have you believe. But wait a second... Isn't Nix, itself, already a sandboxing tool? Why do we need these convoluted semantics and additional complexity leading to a whole-ass [Virtual-Filesystem (VFS) layer](https://github.com/NixOS/nix/pull/6530) that has been in development for years, trying to solve the costs this model introduces? If Nix wanted to enforce purity at evaluation time, couldn't it simply sandbox the process, as it does at build time? We will delve into this a bit more in a later section, but its worth asking.
|
||||
|
||||
Even if you disagree, this is far from the only meaningful boundary Atom introduces. A module in an atom, like a true module should, can only see into its existing scope, even on the file-system level. You see, Atom does copy Nix expressions into the Nix store, just like flakes, but it does so lazily, by virtue of Nix's inherent design. For example, if you need to reference a file inside an atom module, you can do so by referencing it from the modules self-reference: `"${mod}/path-to-file-in-module"`. Only when this file is actually read will the contents of the module directory, not including any submodules or nix files, be copied into the Nix store. If you try to reference the file by relative path, you'll get an error, since the Nix expression itself was copied directly into the Nix store lazily as well, the file doesn't exist relative to its location in it; it must be referenced using the modules systems explicitly outline semantics, or not at all.
|
||||
|
||||
This approach stands in stark contrast to flakes' eager world-copying strategy, which necessitated years of ongoing VFS development to mitigate its costs. By intelligently leveraging Nix's natural laziness, we achieve the same goals without complex VFS machinery. Furthermore, Atoms enforce stricter boundaries than existing Nix organizational systems: the `import` keyword is explicitly forbidden within modules. Instead of allowing arbitrary imports from unknown locations, all code references must flow through the module system itself. This constraint enables future tooling to perform static analysis efficiently, extracting useful information about the code without evaluation, in stark contrast to the current landscape.
|
||||
|
||||
So how do references work in the atom system? If you're up to speed with any modern programming language's module system, you might find it familiar. Similar to Rust's crate system, atoms have a top-level reference `atom` from which every other public member can be referenced, which are denoted by starting with a capital letter. External dependencies are also currently available through here, though this API remains experimental.
|
||||
|
||||
If you need to access private members, you can, through the `pre` scope, which is a reference to the parent module, or `pre.pre` for the grandparent, etc. Anything referenced from `pre` has access to all the private members exported by that module. There is also a recursive reference to the current module: `mod`, and finally, an actual proper scope for a true standard library for the Nix language: the `std` scope. Now if you have used the nix module system before, you might think you have to declare these explicitly as inputs to some kind of functional prototype for every single module.
|
||||
|
||||
Fortunately, no such boilerplate exists. All of these scopes are simply available within the module. This is more important than just providing convenience and a more familiar semantic from other languages, it also allows us to declare our modules and members as the final data structures that we intend them to represent, rather than a prototype of the data, to be constructed after passing arguments to a function. This makes code inside an Atom module more introspective by default. Where one might open a Nix REPL and explore their code full of legacy Nix modules, only to hit an opaque wall when hitting one of these prototypes, which will require a full evaluation to resolve, you can simply continue happily grepping through your code, allowing consumers to more intuitively discern what a library exports, or an atom contains, etc, etc.
|
||||
|
||||
While these features are available today with some effort (see the [README](https://github.com/ekala-project/atom?tab=readme-ov-file)), our ultimate goal is to provide a cohesive system that's intuitively familiar to developers, regardless of their Nix experience. To bridge the gap between our higher-level Atomic module system and the lower-level atom format, we turn to our gateway into the Ekala ecosystem: the `eka` CLI.
|
||||
|
||||
## The Proper Level of Abstraction
|
||||
|
||||
Eka, our CLI frontend, predates even the Atom format it now implements. Rather than following flakes' path of bolting a higher-level interface onto Nix, we approached the problem from first principles: what should a proper interface into a Nix-centric universe look like? This exploration led us to both the Atom format's innovations and several other concepts still in development.
|
||||
|
||||
At its core, `eka` serves as an atomic frontend to a backend service that handles evaluations, builds, and higher-level concerns like deployments and test environments. By decoupling from low-level derivation details, it focuses entirely on providing a clean, intuitive interface to the powerful world of Nix expressions. This design philosophy manifests in several key principles:
|
||||
|
||||
1. Zero-evaluation querying: `eka` should never require Nix evaluation to read basic package information. Versions, dependencies, descriptions, and metadata should all be statically accessible. At most, it needs to know an atom's location, with efficient backend querying capabilities for discovering atoms in the wild.
|
||||
|
||||
2. Static pipeline construction: Building task pipelines, like CI architecture matrices, should be possible without evaluation. These specifications should be readable directly from static manifests, allowing the backend to efficiently schedule work on appropriate machines.
|
||||
|
||||
3. Improved addressing: While flakes introduced useful URI shorthand, we've expanded this concept with Atom URIs. Unlike flakes' hard-coded shortcuts, Atom URIs are configurable, allowing patterns like `work:repo::my-atom@^1`. Crucially, these always expand to full URLs in the final output, ensuring universal understanding while maintaining convenience.
|
||||
|
||||
To support this ambitious scope, we plan to implement a language-agnostic plugin system for `eka`. While the core remains focused on efficient atomic operations and basic backend communication, plugins will extend functionality through a well-defined API surface. This extensibility will become increasingly important as `eka` evolves to help avoid bloat and complexity in the core codebase.
|
||||
|
||||
The ultimate vision for `eka` users is efficient querying of packages, deployment manifests, and configurations across their organization and the open-source landscape—all without upfront Nix evaluation. It should optimize away unnecessary evaluations and builds when artifacts exist in cache, in concert with the backend, proceeding directly to fetching. If `eka` ever needs to perform evaluation for value generation, we've strayed from our design goals.
|
||||
|
||||
While significant work remains, our roadmap is tracked in the [README](https://github.com/ekala-project/eka?tab=readme-ov-file). We're approaching a crucial milestone with the Atom lock format's finalization. Once complete, users will be able to create, link, and depend on Atoms with familiar commands like `eka add my-repo::my-dep@^1.0`—no esoteric knowledge required.
|
||||
|
||||
`eka` represents more than just a CLI tool—it's the gateway into a new paradigm of store-based system interaction. Its role as a frontend is deliberate, allowing it to focus on providing an intuitive interface while delegating complex evaluations and builds to a more sophisticated backend. This separation of concerns brings us to perhaps our most ambitious vision within the Ekala ecosystem: the Eos API & scheduler.
|
||||
|
||||
## A New Dawn
|
||||
|
||||
While we've introduced atoms and their immediate benefits, we've only scratched the surface of how they might revolutionize task distribution across machines. Remember our principle: thought first.
|
||||
|
||||
The atom format isn't just a cornerstone of Ekala by coincidence. While its frontend efficiency gains through `eka` are valuable, its true potential emerges when we consider the backend it enables: the Eos HTTP API.
|
||||
|
||||
Think beyond mere Nix builds—which are already cumbersome to manage. Consider evaluations, integrations, deployments, and operational workflows common to Nix environments. Our vision detaches user machines from costly operations, efficiently distributing evaluations, builds, and tasks across a network. This approach treats Nix's operational use cases as first-class concerns, designed from first principles.
|
||||
|
||||
Eos isn't just about distribution—it's about trust. In an era of increasing supply chain attacks, every evaluation, every build, and every artifact must be cryptographically verifiable. By leveraging atoms' inherent verification properties and Nix's reproducible builds, Eos provides end-to-end supply chain integrity without compromising on performance or usability.
|
||||
|
||||
Why an API? As we progress through the 21st century, well-designed APIs have become fundamental to system architecture. But bringing Nix into the modern era is just the start—we aim to push its boundaries. Nix's unique, idempotent properties cannot be fully leveraged without purpose-built tooling and careful abstraction.
|
||||
|
||||
The Eos HTTP API isn't an afterthought or bolt-on solution like many current Nix-based CI systems. It's fundamental to Ekala's design, crafted to leverage the atom format's advantages while remaining unopinionated about higher-level concerns.
|
||||
|
||||
Although this vision is compelling, transparency requires acknowledging that Eos remains our most theoretical component. We're developing a comprehensive whitepaper to specify its details upfront, avoiding costly iterations in code. Our approach is intentionally iterative, beginning with the cornerstone components and building thoughtfully from there.
|
||||
|
||||
Crucially, Eos represents the spark that ignited the entire Ekala effort. It began with a simple question: "What would the perfect Nix derivation distribution mechanism look like?" The answer—a modern API serving a cleanly abstracted, user-centric client—led us to develop the Atom format and its supporting ecosystem.
|
||||
|
||||
## The Road Ahead
|
||||
|
||||
Before diving deeper into Eos, let's reinforce a crucial point about atoms' role in our architecture. We've established why atoms bridge the gap between low-level Nix derivations and higher-level concepts like repositories and versions. This bridge is fundamental to Eos, which relies on atoms' globally unique identities. Each atom's cryptographic identity, determined by just two elements—the repository's root commit hash and the manifest's Unicode name—provides a stable reference point unlike frequently changing derivations.
|
||||
|
||||
This identity system creates powerful possibilities. Need to mark a fundamental code change? Simply rename the atom. Moving to a new repository? The atom automatically gains a distinct identity. These IDs serve as efficient anchor points for Eos, enabling quick curation without centralization or expensive scanning. While `eka` can directly query atoms and versions within known repositories, Eos can track atoms it encounters across the entire ecosystem, providing location information on demand.
|
||||
|
||||
But atoms are just the beginning. Anyone who's worked with Nix's remote build capabilities—whether the legacy system or the newer experimental UI—knows its limitations in distributing work across machines. Eos aims to solve this through intelligent request handling. For public repositories, Eos can fetch directly using its optimized network. For private code, atoms' efficient transfer format (remember: just git trees of existing objects) enables smart, deduplication-aware transfers.
|
||||
|
||||
Think of Eos as a network of machines that you can organize however you want—hide them in your private network, expose them to the world, or mix and match. The beauty is in its flexibility: you're in control of how your builds and evaluations flow through your infrastructure. At its core, Eos is a Nix-centric scheduler handling evaluations, builds, and higher-level tasks like testing and deployment. For example, we're exploring Raft for high-consistency queue synchronization across machines, ensuring resilience against outages.
|
||||
|
||||
While the distributed design is complex, the goal is straightforward: leverage Nix's unique properties and now, the Atom format, to eliminate redundant work across your network. If one machine evaluates but doesn't build, schedule the derivations elsewhere. If something's built but not cached, ensure it reaches long-term storage en route to users. Everything should be garbage-collectable—imagine keeping major releases permanently while cycling out development builds, etc.
|
||||
|
||||
Eos isn't meant to be monolithic. We're planning to integrate components from [Tvix](https://tvix.dev), which reimagines Nix with modern architecture, to simplify the effort significantly. At its simplest, Eos is a distributed HTTP gateway receiving requests from frontends like `eka`, scheduling them across known machines. While the complexity is significant, it's worthwhile only if it fully exploits Nix's idempotent properties for pipeline optimization from the foundation.
|
||||
|
||||
Our vision extends beyond just managing packages—we're building a framework where security, reproducibility, and sanity are fundamental properties, not afterthoughts. In an era of increasing supply chain attacks, Nix & Ekala's combination of cryptographic verification, reproducible builds, and distributed intelligence positions us to tackle these challenges head-on. We're prioritizing integration with existing standards like SBoM, ensuring that every input is tracked, and every output is independently verifiable.
|
||||
|
||||
While a complete Eos scheduler isn't imminent, our journey has already yielded valuable innovations like the Atom format, module system, and the `eka` CLI. Our commitment to "Thought Driven Development" guides us in building tools that respect both users' freedom and intelligence, providing power without sacrificing transparency or independence.
|
||||
|
||||
We invite you to be part of this evolution. Whether you're a seasoned Nix veteran or just curious about the future of software distribution, join us on [Discord](https://discord.gg/AeZkaQqw) or explore our [GitHub Organization](https://github.com/ekala-project). Together, we can build a future where store-based systems are not just theoretically elegant, but practically transformative for developers everywhere.
|
||||
183
content/blog/std-action.md
Normal file
183
content/blog/std-action.md
Normal file
@@ -0,0 +1,183 @@
|
||||
---
|
||||
title: Standard Action
|
||||
description: Do it once, do it right.
|
||||
taxonomies:
|
||||
tags:
|
||||
- std
|
||||
- nix
|
||||
- devops
|
||||
- github actions
|
||||
author: Tim D
|
||||
authorGithub: nrdxp
|
||||
authorImage: https://avatars.githubusercontent.com/u/34083928?v=4
|
||||
authorTwitter: nrdxp52262
|
||||
date: "2022-12-09"
|
||||
category: dev
|
||||
---
|
||||
|
||||
## CI Should be Simple
|
||||
|
||||
As promised in the [last post](./std), I'd like to expand a bit more on what we've
|
||||
been working on recently concerning Nix & Standard in CI.
|
||||
|
||||
At work, our current GH action setup is rather _ad hoc_, and the challenge of optimizing that path
|
||||
around Nix’s strengths lay largely untapped for nearly a year now. Standard has helped somewhat
|
||||
to get things organized, but there has been a ton of room for improvement in the way tasks are
|
||||
scheduled and executed in CI.
|
||||
|
||||
[Standard Action][action] is our answer. We have taken the last several months of brainstorming
|
||||
off and on as time allows, experimenting to find a path that is versatile enough to be useful
|
||||
in the general case, yet powerful enough for organizations who need extra capacity. So without
|
||||
any further stalling, let's get into it!
|
||||
|
||||
## The Gist
|
||||
|
||||
The goal is simple, we want a CI system that only does work once and shares the result from there.
|
||||
If it has been built or evaled before, then we want to share the results from the previous run
|
||||
rather than start from scratch.
|
||||
|
||||
It is also useful to have some kind of metadata about our actions, which we can use to build
|
||||
matrices of task runners to accomplish our goals. This also allows us to schedule builds on
|
||||
multiple OS trivially, for example.
|
||||
|
||||
Task runners shouldn't have to care about Nix evaluation at all, they should just be able to get
|
||||
to work doing whatever they need to do. If they have access to already reified derivations, they
|
||||
can do that.
|
||||
|
||||
So how can we accomplish this? Isolate the evaluation to its own dedicated "discovery" phase, and
|
||||
share the resulting /nix/store and a json list describing each task and its target derivations.
|
||||
|
||||
From there it's just a matter of opimizing the details based on your usecase, and to that end we
|
||||
have a few optional inputs for things like caching and remote building, if you are so inclined.
|
||||
|
||||
But you can do everything straight on the runner too, if you just need the basics.
|
||||
|
||||
## How it Works
|
||||
|
||||
Talking is fine, but code is better. To that end, feel free to take a look at my own personal CI
|
||||
for my NixOS system and related packages: [nrdxp/nrdos/ci.yml][nrdos].
|
||||
|
||||
What is actually evaluated during the discovery phase is determined directly in the
|
||||
[flake.nix][ci-api].
|
||||
|
||||
I am not doing anything fancy here at the moment, just some basic package builds, but that is
|
||||
enough to illustrate what's happening. You can get a quick visual by look at the summary of
|
||||
a given run: [nrdxp/nrdos#3644114900](https://github.com/nrdxp/nrdos/actions/runs/3644114900).
|
||||
|
||||
You could have any number of matrices here, one for publishing OCI images, one for publishing
|
||||
documentation, one for running deployments against a target environment, etc, etc.
|
||||
|
||||
Notice in this particular example that CI exited in 2 minutes. That's because everything
|
||||
represented by these builds is already cached in the specified action input `cache`, so no work is
|
||||
required, we simply report that the artifacts already exist and exit quickly.
|
||||
|
||||
There is a run phase that typically starts after this build step which runs the Standard action,
|
||||
but since the "build" actions only duty is building, it is also skipped here.
|
||||
|
||||
This is partially enabled by use of the GH action cache. The cache key is set using the following
|
||||
format: [divnix/std-action/discover/action.yml#key][key]. Coupled with the guarantees nix already
|
||||
gives us, this is enough to ensure the evaluation will only be used on runners using a matching OS,
|
||||
on a matching architecture and the exact revision of the current run.
|
||||
|
||||
This is critical for runners to ensure they get an exact cache hit on start, that way they pick
|
||||
up where the discovery job left off and begin their build work immediately, acting directly
|
||||
on their target derivation file instead of doing any more evaluation.
|
||||
|
||||
## Caching & Remote Builds
|
||||
|
||||
Caching is also a first class citizen, and even in the event that a given task fails (even
|
||||
discovery itself), any of its nix dependencies built during the process leading up to that failure
|
||||
will be cached, making sure no nix build _or_ evaluation is ever repeated. The user doesn't have
|
||||
to set a cache, but if they do, they can be rest assured their results will be well cached, we
|
||||
make a point to cache the entire build time closure, and not just the runtime closure, which is
|
||||
important for active developement in projects using a shared cache.
|
||||
|
||||
The builds themselves can also be handed off to a more powerful dedicated remote builder. The
|
||||
action handles remote builds using the newer and more efficient remote store build API, and when
|
||||
coupled with a special purpose service such as [nixbuild.net](https://nixbuild.net), which your
|
||||
author is already doing, it becomes incredibly powerful.
|
||||
|
||||
To get started, you can run all your builds directly on the action runner, and if that becomes
|
||||
a burden, there is a solid path available if and when you need to split out your build phase to a
|
||||
dedicated build farm.
|
||||
|
||||
## Import from What?
|
||||
|
||||
This next part is a bit of an aside, so feel free to skip, but the process outlined above just so
|
||||
happened to solve an otherwise expensive problem for us at work, outlining how thinking through
|
||||
these problems carefully has helped us improve our process.
|
||||
|
||||
IOG in general is a bit unique in the Nix community as one of the few heavy users of Nix’s IFD
|
||||
feature via our [haskell.nix][haskell] project. For those unaware, IFD stands for
|
||||
"import from derivation" and happens any time the contents of some file from one derivations output
|
||||
path is read into another during evaluation, say to read a lock file and generate fetch actions.
|
||||
|
||||
This gives us great power, but comes at a cost, since the evaluator has to stop and build the
|
||||
referenced path if it does not already exist in order to be able to read from it.
|
||||
|
||||
For this reason, this feature is banned from inclusion in nixpkgs, and so the tooling used there
|
||||
(Hydra, _et al._) is not necessarily a good fit for projects that do make use of IFD to some extent.
|
||||
|
||||
So what can be done? Many folks would love to improve the performance of the evaluator itself, your
|
||||
author included. The current Nix evaluator is single threaded, so there is plenty of room for
|
||||
splitting this burden across threads, and especially in the case of IFD, it could theoretically
|
||||
speed things up a great deal.
|
||||
|
||||
However, improving the evaluator performance itself is actually a bit of a red herring as far as
|
||||
we are concerned here. What we really want to ensure is that we never pay the cost of any given Nix
|
||||
workload more than once, no matter how long it takes. Then we can ensure we are only ever
|
||||
building on what has already been done; an additive process if you will. Without careful
|
||||
consideration of this principle beforehand, even a well optimized evaluator would be wasting cycles
|
||||
doing the same evals over and over. There is the nix flake evalulation cache, but it comes with
|
||||
a few [caveats][4279] on its own and so doesn't currently solve our problem either.
|
||||
|
||||
To give you some numbers, to run a fresh eval of my current project at work takes 35 minutes from a
|
||||
clean /nix/store, but with a popullated /nix/store from a previous run it takes only 2.5 minutes.
|
||||
Some of the savings is eaten up by data transfer and compression, but the net savings are still
|
||||
massive.
|
||||
|
||||
I have already begun brainstorming ways we could elimnate that transfer cost entirely by introducing
|
||||
an optional, dedicated [evaluation store](https://github.com/divnix/std-action/issues/10) for those
|
||||
who would benefit from it. With that, there is no transfer cost at all during discovery, and the
|
||||
individual task runners only have to pull the derivations for their particular task, instead of the
|
||||
entire /nix/store produced by discovery, saving a ton of time in our case.
|
||||
|
||||
Either way, this is a special case optimization, and for those who are content to stick with the
|
||||
default of using the action cache to share evaluation results, it should more than suffice in the
|
||||
majority of cases.
|
||||
|
||||
## Wrap Up
|
||||
|
||||
So essentially, we make due with what we have in terms of eval performance, focus on ensuring we
|
||||
never do the same work twice, and if breakthroughs are made in the Nix evaluator upstream at some
|
||||
point in the future, great, but we don't have to wait around for it, we can minimize our burden
|
||||
right now by thinking smart. After all, we are not doing Nix evaluations just for the sake of it,
|
||||
but to get meaningful work done, and doing new and interesting work is always better than repeating
|
||||
old tasks because we failed to strategize correctly.
|
||||
|
||||
If we do ever need to migrate to a more complex CI system, these principles themeselves are all
|
||||
encapsulated in a few fairly minimal shell scripts and could probably be ported to other
|
||||
systems without incredible effort. Feel free to take a look at the source to see what's really
|
||||
goin on: [divnix/std-action](https://github.com/divnix/std-action).
|
||||
|
||||
There are some places where we could use some [help][7437] from [upstream][2946], but even then, the
|
||||
process is efficient enough to be a massive improvement, both for my own personal setup, and for
|
||||
work.
|
||||
|
||||
As I mentioned in the previous post though, Standard isn't just about convenience or performance,
|
||||
but arguable the most important aspect is to assist us in being _thorough_. To ensure all
|
||||
our tasks are run, all our artifacts are cached and all our images are published is no small feat
|
||||
without something like Standard to help us automate away the tedium, and thank goodness for that.
|
||||
|
||||
For comments or questions, please feel free to drop by the official Standard [Matrix Room][matrix]
|
||||
as well to track progress as it comes in. Until next time...
|
||||
|
||||
[action]: https://github.com/divnix/std-action
|
||||
[haskell]: https://github.com/input-output-hk/haskell.nix
|
||||
[nrdos]: https://github.com/nrdxp/nrdos/blob/master/.github/workflows/ci.yml
|
||||
[key]: https://github.com/divnix/std-action/blob/6ed23356cab30bd5c1d957d45404c2accb70e4bd/discover/action.yml#L37
|
||||
[7437]: https://github.com/NixOS/nix/issues/7437
|
||||
[3946]: https://github.com/NixOS/nix/issues/3946#issuecomment-1344612074
|
||||
[4279]: https://github.com/NixOS/nix/issues/4279#issuecomment-1343723345
|
||||
[matrix]: https://matrix.to/#/#std-nix:matrix.org
|
||||
[ci-api]: https://github.com/nrdxp/nrdos/blob/66149ed7fdb4d4d282cfe798c138cb1745bef008/flake.nix#L66-L68
|
||||
218
content/blog/std.md
Normal file
218
content/blog/std.md
Normal file
@@ -0,0 +1,218 @@
|
||||
---
|
||||
title: From DevOS to Standard
|
||||
description: Why we made Standard, and what it has done for us.
|
||||
taxonomies:
|
||||
tags:
|
||||
- std
|
||||
- nix
|
||||
- devops
|
||||
author: Tim D
|
||||
authorGithub: nrdxp
|
||||
authorImage: https://avatars.githubusercontent.com/u/34083928?v=4
|
||||
authorTwitter: nrdxp52262
|
||||
date: "2022-10-31"
|
||||
category: dev
|
||||
---
|
||||
|
||||
## Update: A Video is Worth 1000 Blogs
|
||||
|
||||
For those who would rather watch than read, a colleague of mine has whipped up a great video series
|
||||
exploring Standard in depth, so drop by the [media secition](../media) for links.
|
||||
|
||||
## Two years later...
|
||||
|
||||
DevOS started as a fun project to try and get better with Nix and understand this weird new thing
|
||||
called flakes. Since then and despite their warts, Nix flakes have experienced widespread use, and
|
||||
rightfully so, as a mechanism for hermetically evaluating your system & packages that fully locks
|
||||
your inputs and guarantees you some meaningful level of sanity over your artifacts.
|
||||
|
||||
Yet when I first released it, I never even imagined so many people would find DevOS useful, and I
|
||||
have been truly humbled by all the support and contributions that came entirely spontaneously to the
|
||||
project and ultmately culminated in the current version of [digga][digga], and the divnix org that
|
||||
maintains it.
|
||||
|
||||
## Back to Basics
|
||||
|
||||
For whatever reason, it really feels like time to give a brief update of what has come of this
|
||||
little community experiment, and I'm excited to hopefully clear up some apparent confusion, and
|
||||
hopefully properly introduce to the world [Standard](https://github.com/divnix/std).
|
||||
|
||||
DevOS was never meant to be an end all be all, but rather a heavily experimental sketch while
|
||||
I stumbled along to try and organize my Nix code more effectively. With Standard, we are able to
|
||||
distill the wider experience of some of its contributors, as well as some new friends, and design
|
||||
something a little more focused and hopefully less magical, while still eliminating a ton of
|
||||
boilerplate. Offering both a lightly opinionated way to organize your code into logically typed
|
||||
units, and a mechanism for defining "standard" actions over units of the same type.
|
||||
|
||||
Other languages make this simple by defining a module mechanism into the language where users are
|
||||
freed from the shackles of decision overload by force, but Nix has no such advantage. Many people
|
||||
hoped and even expected flakes to alleviate this burden, but other than the schema Nix expects
|
||||
over its outputs, it does nothing to enforce how you can generate those outputs, or how to organize
|
||||
the logical units of code & configuration that generate them.
|
||||
|
||||
## A Departure from Tradition
|
||||
|
||||
It is fair to say that the nixpkgs module system has become the sort of "goto" means of managing
|
||||
configuration in the Nix community, and while this may be good at the top-level where a global
|
||||
namespace is sometimes desirable, it doesn't really give us a generic means of sectioning off our
|
||||
code to generate both configuration _and_ derivation outputs quickly.
|
||||
|
||||
In addition to that, the module system is fairly complex and is a bit difficult to anticate the
|
||||
cost of ahead of time due to the fixed-point. The infamous "infinite traces" that can occur during
|
||||
a Nix module evaluation almost never point to the actual place in your code where the error
|
||||
originates, and often does even contain a single bit of code from the local repository in the trace.
|
||||
|
||||
Yet as the only real game in town, the module system has largely "de facto" dictated the nature
|
||||
of how we organize our Nix code up til now. It lends itself to more of a "depth first" approach
|
||||
where modules can recurse into other modules ad infinitum.
|
||||
|
||||
## A Simpler Structure
|
||||
|
||||
Standard, in contrast, tries to take an alternative "breadth first" approach, ecouraging code
|
||||
organization closer to the project root. If true depth is called for, flakes using Standard can
|
||||
compose gracefully with other flakes, whether they use Standard or not.
|
||||
|
||||
It is also entirely unopionated on what you output, there is nothing stopping you from simply
|
||||
exporting NixOS modules themselves, for example, giving you a nice language level
|
||||
compartmentalization strategy to help manager your NixOS, Home Manager or Nix Darwin configurations.
|
||||
|
||||
Advanced users may even write their own types, or even extend the officially supported ones. We
|
||||
will expand more on this in a later post.
|
||||
|
||||
But in simple terms, why should we bother writing the same script logic over and over when we can be
|
||||
guaranteed to recieve an output of a specific type, which guarantees any actions we define for the
|
||||
type at large will work for us: be it deploying container images, publishing sites, running
|
||||
deployments, or invoking tests & builds.
|
||||
|
||||
We can ensure that each image, site, or deployment is tested, built, deployed and published in
|
||||
a sane and well-defined way, universally. In this way, Standard is meant to not only be convenient,
|
||||
but comprehensive, which is an important property to maintain when codebases grow to non-trivial
|
||||
size.
|
||||
|
||||
There is also no fixed-point so, anecdotably, I have yet to hit an eval error in Standard based
|
||||
projects that I couldn't quickly track down; try saying that about the module system.
|
||||
|
||||
## A CLI for productivity
|
||||
|
||||
The Nix cli can sometimes feel a little opaque and low-level. It isn't always the best interface
|
||||
to explain and explore what we can actually _do_ with a given project. To address this issue in
|
||||
a minimal and clean way, we package a small go based cli/tui combo to quickly answer exactly this
|
||||
question, "What can I do with this project?".
|
||||
|
||||
This interface is entirely optional, but also highly useful and really rather trivial thanks to a
|
||||
predicatable structure and well typed outputs given to us in the Nix code. The schema for anything
|
||||
you can do follows the same pattern: "std //$cell/$block/$target:$action". Here the "cell" is the
|
||||
highest level "unit", or collection of "blocks", which are well-typed attribute sets of "targets"
|
||||
sharing a colleciton of common "actions" which can be performed over them.
|
||||
|
||||
### At a Glance
|
||||
|
||||
The TUI is invaluable for quickly getting up to speed with what's available:
|
||||
|
||||
```console
|
||||
┌────────────────────────────────────────────────────────────────────────────────┐┌───────────────────────────────────┐
|
||||
│| Target ││ Actions │
|
||||
│ ││ │
|
||||
│ 176 items │││ build │
|
||||
│ │││ build this target │
|
||||
│ //automation/packages/retesteth ││ │
|
||||
│ testeth via RPC. Test run, generation by t8ntool protocol ││ run │
|
||||
│ ││ exec this target │
|
||||
││ //automation/jobs/cardano-db-sync ││ │
|
||||
││ Run a local cardano-db-sync against our testnet ││ │
|
||||
│ ││ │
|
||||
│ //automation/jobs/cardano-node ││ │
|
||||
│ Run a local cardano-node against our testnet ││ │
|
||||
│ ││ │
|
||||
|
||||
```
|
||||
|
||||
## A Concise Show & Tell
|
||||
|
||||
The central component of Standard is the cell block API. The heirarchy is "cell"→"block", where
|
||||
we defined the individual block types and names directly in the flake.nix.
|
||||
|
||||
The function calls in the "cellBlocks" list below are the way in which we determine which "actions"
|
||||
can be run over the contents of the given block.
|
||||
|
||||
```nix
|
||||
# flake.nix
|
||||
{
|
||||
inputs.std.url = "github:divnix/std";
|
||||
outputs = inputs: inputs.std.growOn {
|
||||
inherit inputs;
|
||||
systems = ["x86_64-linux"];
|
||||
# Every file in here should be a directory, that's your "cell"
|
||||
cellsFrom = ./nix;
|
||||
# block API declaration
|
||||
cellBlocks = [
|
||||
(std.functions "lib")
|
||||
(std.installables "packages")
|
||||
(std.devshells "devshells")
|
||||
];
|
||||
};
|
||||
}
|
||||
|
||||
# ./nix/dev/packages.nix
|
||||
# nix build .#$system.dev.packages.project
|
||||
# std //dev/packages/project:build
|
||||
{
|
||||
inputs, # flake inputs with the `system` abstracted, but still exposed when required
|
||||
cell # reference to access other blocks in this cell
|
||||
}: let
|
||||
inherit (inputs.nixpkgs) pkgs;
|
||||
in
|
||||
{
|
||||
project = pkgs.stdenv.mkDerivation {
|
||||
# ...
|
||||
};
|
||||
}
|
||||
|
||||
# ./nix/automation/devshells/default.nix
|
||||
# nix develop .#$system.dev.devshells.dev
|
||||
# std //automation/devshells/dev:enter
|
||||
{
|
||||
inputs,
|
||||
cell
|
||||
}: let
|
||||
inherit (inputs) nixpkgs std;
|
||||
inherit (nixpkgs) pkgs;
|
||||
# a reference to other cells in the project
|
||||
inherit (inputs.cells) packages;
|
||||
in
|
||||
{
|
||||
dev = std.mkShell { packages = [packages.project]; };
|
||||
}
|
||||
```
|
||||
|
||||
## Encouraging Cooperation
|
||||
|
||||
Standard has also given us a useful mechanism for contributing back to upstream where it makes
|
||||
sense. We are all about maintaining well-defined boundaries, and we don't want to reimplement the
|
||||
world if the problem would be better solved elsewhere. Work on Standard has already led to several
|
||||
useful contributions to both nixpkgs and even a few in nix proper, as well as some in tangentially
|
||||
related codebases, such as github actions and go libraries.
|
||||
|
||||
One very exciting example of this cooperation is the effort we've expended integrating
|
||||
[nix2container][n2c] with Standard. The work has given us insights and position to begin defining an
|
||||
officially supported specification for [OCI images][oci] built and run from Nix store paths, which
|
||||
is something that would be a huge win for developers everywhere!
|
||||
|
||||
We believe interoperability with existing standards is how Nix can ultimately cement itself into
|
||||
the mainstream, and in a way that is unoffensive and purely additive.
|
||||
|
||||
## CI simplified
|
||||
|
||||
Instead of making this a mega post, I'll just leave this as a bit of a teaser for a follow-up post
|
||||
which will explore our recent efforts to bring the benefits Standard to GitHub Actions _a la_
|
||||
[std-action][action]. The target is a Nix CI system that avoids ever doing the same work more than
|
||||
once, whether its evaluating or building, and versatile enough to work from a single user project
|
||||
all the way up to a large organization's monorepo. Stay tuned...
|
||||
|
||||
[digga]: https://github.com/divnix/digga
|
||||
[nosys]: https://github.com/divnix/nosys
|
||||
[action]: https://github.com/divnix/std-action
|
||||
[grow]: https://std.divnix.com/guides/growing-cells.html
|
||||
[harvest]: https://github.com/divnix/std/blob/main/src/harvest.nix
|
||||
[n2c]: https://github.com/nlewo/nix2container
|
||||
[oci]: https://github.com/opencontainers/image-spec/issues/922
|
||||
5
content/projects/_index.md
Normal file
5
content/projects/_index.md
Normal file
@@ -0,0 +1,5 @@
|
||||
+++
|
||||
title = "Projects"
|
||||
sort_by = "weight"
|
||||
template = "cards.html"
|
||||
+++
|
||||
9
content/projects/atom.md
Normal file
9
content/projects/atom.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "atom"
|
||||
description = "A deterministic source packaging format built on Git's object model."
|
||||
weight = 0
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/ekala-project/atom"
|
||||
+++
|
||||
9
content/projects/atomix.md
Normal file
9
content/projects/atomix.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "atomix"
|
||||
description = "A minimal, performant module system for Nix code."
|
||||
weight = 0
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/ekala-project/atom/tree/master/atom-nix"
|
||||
+++
|
||||
9
content/projects/bitte.md
Normal file
9
content/projects/bitte.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "bitte"
|
||||
description = "Nix Ops for Terraform, Consul, Vault, Nomad."
|
||||
weight = 3
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/input-output-hk/bitte"
|
||||
+++
|
||||
9
content/projects/cfdyndns.md
Normal file
9
content/projects/cfdyndns.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "cfdyndns"
|
||||
description = "CloudFlare Dynamic DNS Client."
|
||||
weight = 2
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/nrdxp/cfdyndns"
|
||||
+++
|
||||
9
content/projects/digga.md
Normal file
9
content/projects/digga.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "digga"
|
||||
description = "A Nix flake utility library for managing nixos, hm and devshells"
|
||||
weight = 3
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://digga.divnix.com/"
|
||||
+++
|
||||
9
content/projects/eka.md
Normal file
9
content/projects/eka.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "eka"
|
||||
description = "An atomic, plugin-based CLI frontend to the Eos API."
|
||||
weight = 0
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/ekala-project/eka"
|
||||
+++
|
||||
9
content/projects/eos.md
Normal file
9
content/projects/eos.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "eos"
|
||||
description = "A work-in-progress distributed HTTP scheduler designed for store-driven build systems."
|
||||
weight = 0
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/ekala-project"
|
||||
+++
|
||||
9
content/projects/ethic.md
Normal file
9
content/projects/ethic.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "hackers-ethic"
|
||||
description = "First, hinder no thought: A Code of Ethics for Digital Freedom."
|
||||
weight = 0
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/EthicsCodes/hackers-ethic"
|
||||
+++
|
||||
9
content/projects/home.md
Normal file
9
content/projects/home.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "home"
|
||||
description = "Ekala based dotfiles leveraging Nix and home-manager."
|
||||
weight = 1
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/nrdxp/home"
|
||||
+++
|
||||
9
content/projects/nixpkgs.md
Normal file
9
content/projects/nixpkgs.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "nixpkgs"
|
||||
description = "ex-maintainer & NixOS release manager."
|
||||
weight = 2
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/NixOS/nixpkgs"
|
||||
+++
|
||||
9
content/projects/nosys.md
Normal file
9
content/projects/nosys.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "nosys"
|
||||
description = "A hassle free Nix flake system handler library."
|
||||
weight = 1
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/divnix/nosys"
|
||||
+++
|
||||
9
content/projects/pc.md
Normal file
9
content/projects/pc.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "partnerchain"
|
||||
description = "A Substrate-based blockchain running on top of the Cardano network."
|
||||
weight = 0
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/input-output-hk/partner-chains"
|
||||
+++
|
||||
9
content/projects/rfc-175.md
Normal file
9
content/projects/rfc-175.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "RFC 175"
|
||||
description = "A proposal to enhance moderation practices within the Nix community."
|
||||
weight = 1
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://github.com/nrdxp/rfcs/blob/rfc-175/rfcs/0175-appeals-council.md"
|
||||
+++
|
||||
9
content/projects/standard.md
Normal file
9
content/projects/standard.md
Normal file
@@ -0,0 +1,9 @@
|
||||
+++
|
||||
title = "standard"
|
||||
description = "A Nix-centric devops framework."
|
||||
weight = 1
|
||||
|
||||
|
||||
[extra]
|
||||
link_to = "https://std.divnix.com/"
|
||||
+++
|
||||
Reference in New Issue
Block a user