As I get older, I realize there are problems in life that you cannot solve. I’m not saying that package management is an impossible problem, but it is more so like politics or nature. Package management is the art of looking for trouble, finding it whether it exists or not, diagnosing it incorrectly, and applying the wrong remedy. Or at least this is how it is perceived by the outside world. Look, I get it. Everything you believe a package manager should do and seems like common sense, they end up doing the opposite. Why is this? Well it is a matter of trust.
I am not an expert(if such thing even exists) but have worked in the space for long enough where I feel an opinion can be shared publicly.
Note: I use various words in package management throughout this post. I am primarily referring to the entire package management ecosystem (package registry, packages, authors, tooling, processes, OSS, companies, etc).
First, an analogy
Anytime you want to do any small thing by installing a package, there is a lot of risk. Think of it this way…
We download code
From the internet
Written by unknown individuals
That we haven’t read
Which we execute
With full permissions
On our trusted devices
Where we keep our most important data
Each line above represents a layer of trust with its own unique problems in package management. Today, there is really no other option but to trust.
Therefore it is a miracle that this all works.
There’s a partnership that many tend to oversee — symbiosis, which is Greek for ‘together’ and ‘living’. The word itself is a neutral one, implying any form of coexistence. If any partner benefits or nobody benefits, it falls under this rubric of symbiosis. Almost every major partnership in our natural world is like this. Cheats are always a problem. Betrayal lurks perpetually on the horizon. Groups may work well together, but if one can get the same benefits without spending as much energy or effort, they will do so unless punished or policed. H. G. Wells wrote about this in 1930:
“Every symbiosis is in its degree underlain by hostility, and only by proper regulation and often elaborate adjustment, can the state of mutual benefit be maintained. Even in human affairs, partnerships for mutual benefit are not so easily kept up, in spite of men being endowed with intelligence and so being able to grasp the meaning of such a relation. But in lower organisms, there is no such comprehension to help keep the relationship going. Mutual partnerships are adaptations as blindly entered into and as unconsciously brought about as any others.”
Now we need to separate important from harmonious. The package ecosystem is incredibly important, but it doesn’t mean that it’s always harmonious. Involved partners may benefit from this inherent tension, but symbiosis is conflict by nature — conflict that can never be totally resolved. We can contain this conflict however. Think of it like agriculture. We use fences and barriers to mark the boundaries of our gardens. We use fertilizer to feed the plants. We uproot and poison incipient weeds. And we set the garden in a place with the right temperature, soil, and levels of sunlight to nourish whatever we want to grow. When it comes to package management, we use equivalents of all of these measures to set the terms and conditions for these partnerships. We meet each one in turn.
Conflict as we are relating it to a package ecosystem is no different than a typical prisoner’s dilemma. Although there’s no best strategy, a strong strategy might be “tit for tat” which generally means to cooperate in the first round, and after that, you do whatever the other player did in the previous round. It’s quite a complementary strategy to our “trust, but verify” world, by ensuring the other party meets their obligations. Those who both play a tit for tat will always cooperate. Those who play tit for tat and those who always cooperate will always cooperate. There aren’t major losses this way even for those who choose to retaliate.
When there is a signaling error however in a tit for tat strategy due to a misunderstanding or lack of communication, it can cause a loss of trust. The discoverability of this vulnerability provided two new variations of tit for tat. The first being “contrite tit for tat” which retaliates only if the other side has defected twice in a row. The second “forgiving tit for tat” which forgives one third of the defections. Both continue to avoid doomsday signal-error scenarios but are still vulnerable to exploitation. Transitioning from a contrite variation to a forgiving variation does one major thing however. It re-establishes trust with time. Both humans and animals have systems of reciprocity with sensitivity to cheating, but you may ask how can cooperation ever start in the first place? The answer is simple. The always or sometimes defect-ers either must cooperate or go extinct. There’s really no choice but to live together.
Now there’s a number of trust factors in which conflict arises. Here’s a few that come to mind:
Security. People trust that the packages they download and install through a package manager are free from malicious code, security vulnerabilities, and unexpected behavior.
Integrity. People trust that the packages they receive have not been opened or tampered with in any way during transit.
Reliability. People trust that the packages they request or send will be consistent and reliably delivered.
Maintainer Integrity. People rely on maintainers to provide accurate, up-to-date, and secure packages.
Reputation. People are drawn to a strong reputation for security and reliability, thus fostering a healthy software ecosystem.
Community. People trust that their contributions will be reviewed, tested, and integrated into various packages and people trust that their reports to the registry will be reviewed and acted upon in due time.
Compliance. People trust that licensing and legal requirements are enforced through policies related to the terms of service.
When any of these obligations are broken, there is a signaling error causing a state of distrust in the relationship.
No bad or good
There really is no “good” nor “bad” partners when it comes to trust in package management, rather just states of trust. The closest comparison would be to that of microbes. Labeling does no good to bacteria given they can be everything and anything(parasitic, mutualistic, etc) based on the context of the symbiotic relationship.
It would be better to use terms describing states of being like “cooperating” or “defecting”. These terms of “good” or “bad” belong in children’s stories because they are ill-suited for describing the messy, fractious, contextual relationships of the package management world.
If we could move past these labels, perhaps we can see that trust doesn’t always have to be a “good” thing. But that trust can just be what it is. A complex relationship between things.
We talked at a high level of the “impossible” challenge that is package management. We dived into an example demonstrating the inherent trust everyone gives to be apart of the ecosystem. Next, we talked about the symbiosis nature of package management and walked through optimal strategies of trust and regaining trust. Finally, we went through various factors that help signal trust in the relationship and how there’s really no good or bad when it comes to labeling trusting relationships.