35 stories
·
2 followers

Wednesday, November 29 - self-hosting, cloud disentanglement, windmill tilting, etc.

1 Comment

Wednesday, November 29

self-hosting, cloud disentanglement, windmill tilting, etc.

So my computational life is kind of a mess, and also is more locked-in to services provided by third-party corporations (subtype: gigantic, evil) than I’d like. I’ve spent years promising myself I’d become less dependent on the megacorps, but I’m still beholden to Google and a pack of others.

I’ve decided that late 2017 is as good a time as any to start working on this problem in earnest.

This is the kind of list that I’d normally write on a piece of paper with checkboxes, but I may as well document it here in case it’s useful to anyone else.

These are the systems I’m concerned with:

mail

I’ve been on GMail since August of 2004 (I had to check this), when it was still an invite-your-friends service working to build clout by starting with nerds. I have about 9 gigs worth of archives; by volume most of that is probably mailing lists, notifications, and other machine-driven noise, but there’s plenty I’d like to retain. Right now I plan to keep this locally in Maildir form and use some kind of desktop client for everything.

There’s also the basic problem of all the other identities that get attached to an e-mail address in the course of thirteen years of heavy use. I can’t afford to delete the account in the short term, but I guess I can forward everything and spend months chipping away at all the stuff tied to it.

Difficulty: Unpleasant, repetitive, but well understood and achievable.

phone service

I’ve had the same phone number for most of my adult life. I’m not sure whether it’s possible to pry it out of the clutches of Verizon, but I’d like to. Even if that’s not possible, things I want Verizon to lose include:

  • constant knowledge of my location
  • control over the OS and installed applications on my phone
  • interception of most of my data traffic

I’m aware that any connection to the phone network will involve an unsavory corporate provider — I just have a special and particular contempt for Verizon, built on years of acquaintance.

Difficulty: Heroic levels of cat vacuuming, probably.

mobile os, apps, etc.

I use an Android device. At its best, Android is a pretty reasonable user experience, but it’s full of tracky shit and increasingly pushy about integrating itself with the broader panopticon. Every time my phone asks me to review and post photos of some random gas station off of I-25, or encourages to me to read a news story “based on your interest in Donald Trump”, I feel incrementally more alienation and loathing for everything our technical culture has become.

Since alienation and loathing are no fun, I would like to stop using Android (preferably without switching to iOS).

Difficulty: See above, re: cat vacuuming. If this is achievable, it’s probably in part by splitting the functions of a phone out into a couple of devices and just abandoning others.

e-books

I’m on my third or fourth Kindle, the e-ink kind. It’s convenient enough, but Amazon is doing its best to eat everything (including the publishing industry), and I would like to contribute less energy to their efforts. Plus I’m pretty sure future iterations of any Amazon device will eventually include an always-on microphone, and I am not interested.

Difficulty: I think there’s other hardware out there, and other marketplaces for e-books. Also, paper mostly still has better ergonomics, aside from weight and bulk. But then most of my book reading happens in an easy chair or a bathtub, not on an airplane.

laptop and desktop hardware

Some set of world-historically stupid assholes at Intel decided that it would be a good idea to install a full-blown operating system completely outside of end-user control on most of the chipsets they’ve sold for the last decade, so it’s even less possible than naive paranoia would suggest to trust the hardware I own.

Difficulty: Fucked. I can do quite a bit using relatively open single-board ARM machines like the Beaglebone or the Novena, but I can’t easily escape from the need for a system robust enough to run a bunch of modern web apps.

Further notes to come as I tackle these.

Read the whole story
thcipriani
41 days ago
reply
godspeed.
Share this story
Delete

Saturday, November 18 - App::WRT - WRiting Tool, a static site generator and related utilities

2 Shares

Saturday, November 18

App::WRT - WRiting Tool, a static site generator and related utilities

Probably a decade after the first time I put it on a TODO list, I finally got around to publishing this site’s underlying software on CPAN. It didn’t used to be called App::WRT; for a long time it was just display.pl, and then Display.pm when I turned it into (sort of) a library.

Last February, I switched it from CGI that ran server-side on every page request to a site generator that would render the entire site to static HTML files. That July, after agonizing about good command names not already taken by real software, I switched the command-line interface from display to wrt, short for writing tool. CPAN naming guidelines suggest putting this sort of thing in the App namespace, so that’s what I did.

CPAN is the Comprehensive Perl Archive Network, a big repository of libraries, utilities, and documentation in Perl. Which is to say that it’s Perl’s answer to npm, Packagist, RubyGems, PyPI, etc. (It would probably be more accurate to say those things are other languages' answer to CPAN, since CPAN dates to the mid-1990s.)

I’ve generally had a bad experience with language-specific package management systems, but after all these years CPAN remains an exception, for all of its foibles. Publishing a release to CPAN turns out to be a very 1990s / early 2000s kind of experience, with a wait to see results and a generally piecemeal feeling. It suffers by comparison to the “push a git tag to the remote” approach to creating a “release” on GitHub. On the other hand, it pushed me to make a bunch of improvements to the documentation and fill out a handful of the features that wrt needs to be usable as a standalone tool.

🌲

I know no one else will ever use this thing. In case you did want to, installing on most GNU/Linux systems should be as simple as running:

$ sudo cpan -i App::WRT

Or, if you happen to have cpanm installed:

$ sudo cpanm App::WRT

Once installed, you should be able to run wrt:

$ wrt
wrt - a writing tool

Usage: /usr/local/bin/wrt [command] [args]
    wrt init        Initialize a wrt repository
    wrt display     Print HTML for entries
    wrt render-all  Render all defined entries to filesystem
    wrt addprop     Add a property to an entry
    wrt findprop    Find entries containing certain properties
    wrt -h          Print this help message

You must specify a command.

In order to make an entry for the current day, create a file like archives/2017/9/18, and write some HTML in it. Or use Markdown, like so:

<h1>Saturday, November 18</>

<markdown>
Your text here.
</markdown>

If I live long enough, I might get around to rewriting wrt in something else, but aside from C, I’m not sure I could have started out by picking a language more boringly likely than Perl to keep working for a couple of decades.

The underlying archive format could be better in some ways, but so far it’s also been fairly future-proof. My only real worry is that one of these days, as the open web vanishes further into the maw of facegooglemazon, HTML itself may start to seem like a bad idea. In that case, however, it should be pretty easy to convert the simple subset of HTML I’m using here to some other language.

Read the whole story
thcipriani
50 days ago
reply
brennen
58 days ago
reply
Boulder, CO
Share this story
Delete

Monday, October 16 - decades

2 Shares

Monday, October 16

decades

Paul Ford wrote about starting ftrain 20 years ago:

I started this website 20 years ago, give or take a week. The original address was www.interactive.net/~ford. Eventually it migrated here into the form you see. I took it very seriously for many years and it earned me thousands of readers, thousands of emails, and tons of opportunity. It was better at generating opportunity than money. I drifted away for all the regular reasons.

Which got me thinking: The oldest surviving bits of this website go back to April of 1997, so it's 20 years, give or take six months. It lived other places for a while (GeoCities and a shell box with a tilde in the URL) until I actually got a domain for it. p1k3.com was the first domain I ever bought, and I chose it because my middle name is Pike and I like the number 13 and it was four characters long, which even in the early years of this century was getting to be kind of hard to come up with. There was also this running joke with friends from IRC, about whether a pike was a weapon or a fish, and I guess that must have played into my thinking somehow.

p1k3 has clearly not made me into a low-key internet celebrity. I don't know about opportunity, but it has helped me get a couple of good jobs, and probably prevented me from getting several more bad ones. To guess at its current readership, I think that about a dozen humans might see this post sooner or later.

I wrote one possible variation on the post you're reading now back in February — the one where I regret writing so much stupid bullshit. That one doesn't really explain why I've written so much less this year than most, though. This other one where I worry about self-surveillance in an age of weaponized data and network fascists comes a lot closer to the mark.

In 2017, I've fully given up on some things. A big one is the World Wide Web. The "open web", as such, is dead. Or at best on life support. The actually existing web is, mostly, bad. It's an abject failure in the terms I thought I was involved with it on, and its architecture has helped bootstrap an internet that's hostile to my values, if not ultimately to human life itself.

It's no longer possible to use the web in a way that respects your privacy, autonomy, basic personhood, etc. And for the same reasons, it's incredibly difficult to work on the web for a living in any kind of ethical fashion.

But then: So what?

I think it's broadly true that most of us should treat the network as a hostile environment, and that any information we publish about ourselves will be used against us and our communities by systems we have no control over — systems operating under few legal constraints, answering only to the profit motive, under the authority of complete assholes with no sense of responsibility, proportion, or historical perspective.

It's really tempting, in the face of this conviction, to shut up and just focus on sequestering myself from the network to whatever limited extent that's still possible.

On the other hand. Writing is one of the only real powers I've ever had, and the surface of this terrible website is still mine to write on. The web is dead to me, as a hope or a cause, and the world it's made — the world that so many thousands of us helped to make — is in bad shape and getting worse. But why should I give up my only real canvas, the only place where I have any voice at all?

Possibly (almost certainly) having a voice is itself an illusion, irrelevant to the course of things now. But I guess it's something.

Read the whole story
thcipriani
78 days ago
reply
brennen
78 days ago
reply
Boulder, CO
Share this story
Delete

Digital Resource Lifespan

1 Comment and 5 Shares
I spent a long time thinking about how to design a system for long-term organization and storage of subject-specific informational resources without needing ongoing work from the experts who created them, only to realized I'd just reinvented libraries.
Read the whole story
thcipriani
78 days ago
reply
Share this story
Delete
1 public comment
tante
78 days ago
reply
Digital Resource Lifespan
Oldenburg/Germany

@20

1 Comment
Not any kind of eulogy, thanks. And no header image, either.
Read the whole story
thcipriani
91 days ago
reply
"With every flap of their terrible fins they squash another good idea in the interest of consolidating pablum into a single database, the better to jam it down our mental baby duck feeding tubes in order to make even more of the cognitive paté that Silicon Valley is at pains to proclaim a delicacy."

Huh.
Share this story
Delete

Software Is About Storytelling

1 Share

Software engineering is more a practice in archeology than it is in building. As an industry, we undervalue storytelling and focus too much on artifacts and tools and deliverables. How many times have you been left scratching your head while looking at a piece of code, system, or process? It’s the story, the legacy left behind by that artifact, that is just as important—if not more—than the artifact itself.

And I don’t mean what’s in the version control history—that’s often useless. I mean the real, human story behind something. Artifacts, whether that’s code or tools or something else entirely, are not just snapshots in time. They’re the result of a series of decisions, discussions, mistakes, corrections, problems, constraints, and so on.  They’re the product of the engineering process, but the problem is they usually don’t capture that process in its entirety. They rarely capture it at all. They commonly end up being nothing but a snapshot in time.

It’s often the sign of an inexperienced engineer when someone looks at something and says, “this is stupid” or “why are they using X instead of Y?” They’re ignoring the context, the fact that circumstances may have been different. There is a story that led up to that point, a reason for why things are the way they are. If you’re lucky, the people involved are still around. Unfortunately, this is not typically the case. And so it’s not necessarily the poor engineer’s fault for wondering these things. Their predecessors haven’t done enough to make that story discoverable and share that context.

I worked at a company that built a homegrown container PaaS on ECS. Doing that today would be insane with the plethora of container solutions available now. “Why aren’t you using Kubernetes?” Well, four years ago when we started, Kubernetes didn’t exist. Even Docker was just in its infancy. And it’s not exactly a flick of a switch to move multiple production environments to a new container runtime, not to mention the politicking with leadership to convince them it’s worth it to not ship any new code for the next quarter as we rearchitect our entire platform. Oh, and now the people behind the original solution are no longer with the company. Good luck! And this is on the timescale of about five years. That’s maybe like one generation of engineers at the company at most—nothing compared to the decades or more software usually lives (an interesting observation is that timescale, I think, is proportional to the size of an organization). Don’t underestimate momentum, but also don’t underestimate changing circumstances, even on a small time horizon.

The point is, stop looking at technology in a vacuum. There are many facets to consider. Likewise, decisions are not made in a vacuum. Part of this is just being an empathetic engineer. The corollary to this is you don’t need to adopt every bleeding-edge tech that comes out to be successful, but the bigger point is software is about storytelling. The question you should be asking is how does your organization tell those stories? Are you deliberate or is it left to tribal knowledge and hearsay? Is it something you truly value and prioritize or simply a byproduct?

Documentation is good, but the trouble with documentation is it’s usually haphazard and stagnant. It’s also usually documentation of how and not why. Documenting intent can go a long way, and understanding the why is a good way to develop empathy. Code survives us. There’s a fantastic talk by Bryan Cantrill on oral tradition in software engineering where he talks about this. People care about intent. Specifically, when you write software, people care what you think. As Bryan puts it, future generations of programmers want to understand your intent so they can abide by it, so we need to tell them what our intent was. We need to broadcast it. Good code comments are an example of this. They give you a narrative of not only what’s going on, but why. When we write software, we write it for future generations, and that’s the most underestimated thing in all of software. Documenting intent also allows you to document your values, and that allows the people who come after you to continue to uphold them.

Storytelling in software is important. Without it, software archeology is simply the study of puzzles created by time and neglect. When an organization doesn’t record its history, it’s bound to repeat the same mistakes. A company’s memory is comprised of its people, but the fact is people churn. Knowing how you got here often helps you with getting to where you want to be. Storytelling is how we transcend generational gaps and the inevitable changing of the old guard to the new guard in a maturing engineering organization. The same is true when we expand that to the entire industry. We’re too memoryless—shipping code and not looking back, discovering everything old that is new again, and simply not appreciating our lineage.

Read the whole story
thcipriani
98 days ago
reply
Share this story
Delete
Next Page of Stories