Processes and tools
The folks who gave us the Agile Manifesto wrote “We have come to value individuals and interactions over processes and tools.” Agile in 3 Minutes, my micropodcast about doing business effectively, reflects the same valuation. What I’m offering each week is the best of myself I can give. What I’m hoping for each week are more and better interactions between you and the folks you work with.
There are, unavoidably, some processes involved. You may already know how Agile in 3 Minutes began, how it became playable directly within Tweets, and how it gets written and published each week. Now that I can mostly focus on one new episode at a time, I’m trying to maximize the amount of work not done.
And in those processes, I rely on tools. If I didn’t have them, there probably wouldn’t be a show. It’s Thanksgiving in the USA and I’m very thankful to have these tools to hand. I’d like to tell you a bit about them.
Tools I’ve paid for
Writing is hard for me, so I arrange my writing environment carefully. It’s easier for me to stay focused when there’s no way to have lots of open windows and the screen isn’t too big. Current preference: iPad Air with the Anker Bluetooth keyboard.
Episode scripts are plain-text essays with a title, an emphasized word here and there, and the occasional bulleted list. As far as my brain’s concerned, that’s just Markdown. Textastic is a general-purpose text editor for iOS that highlights syntax, counts words, displays rendered Markdown, and syncs with Dropbox. (That’s how I update the book every week with the latest essay.) Lately I’ve been experimenting with Drafts for the unconstrained “writing” phase, reserving Textastic for the highly constrained “editing, editing, editing” phase. I’m liking it.
When it’s time to record, I read into a Blue Yeti USB microphone connected to a MacBook Air. I have a cheap pop filter, but it doesn’t fasten very well to the Yeti and is mildly annoying to travel with, so I haven’t been using it. Instead I’m speaking into the Yeti at a slight angle and trying not to pop my P’s too hard. Know something else I could try? Please comment on this post and let me know.
The Yeti isn’t great to travel with either — it’s large, somewhat heavy, and periodically attracts the attention of TSA inspectors — but the couple episodes I recorded into a Samson GoMic were of noticeably lesser audio quality. The GoMic is very small and will doubtless come in handy, but for Agile in 3 Minutes I quickly returned to the Yeti.
To monitor what’s going into the microphone as I record, I use Bose QuietComfort 25 headphones. The noise-canceling feature is a nice touch in other contexts (such as travel), but while recording, I want to hear any and all hotel noise that might sneak in. I count on the Yeti’s cardioid pattern to filter most of it out.
Open-source tools I use
On the VPS, I run NetBSD, along with hundreds of third-party tools I keep updated with pkgsrc. Audio and text content is stored in Git, from which the website and RSS feed are generated by Ikiwiki. Apache is way overkill for serving up static content, but several other sites on the same VPS were already using it.
I’ve contributed to NetBSD, pkgsrc, and Ikiwiki.
Ways you can contribute to my work
Thanks to the generosity of others, producing Agile in 3 Minutes doesn’t cost me as much as it might. But it does require (aside from the time I willingly put into it) some one-time and running costs I wouldn’t otherwise be carrying. The microphone, I definitely wouldn’t be carrying.
Is Agile in 3 Minutes helping you have more and better interactions at work? I’d love to hear about it. That’s all the gratification I need.
Would you like to share the show with others, give me topic ideas, read the essays at your own pace, or drip-fund the production of new episodes? Then I’ll have even more to be grateful for. Here’s how you can help.
Scope of this article
The 3 things almost every team does to meet deadlines that make things worse:
1. Hire more developers
2. Cut corners
3. Work long hours
I must admit, my deadline-missing expertise is fairly wide-ranging. But this piece is specifically about deadlines that arise when we undertake to develop software, and how you can do your level best to miss them.
Don’t think too hard about whether they’re needed. When in doubt, set one.
Set deadlines unilaterally
Avoid taking input from the customer who depends on the outcome or the team trying to deliver it. Take your best guess and call it good.
Disagree on “deadline”
Some possible meanings of the word “deadline”:
- If we miss, we go out of business
- If we miss, our customer goes out of business
- If we miss, my manager will blame me and/or I’ll be fired
- If we miss, I’ll blame you and/or you’ll be fired
- I don’t think you’ve been working hard enough lately
These meanings aren’t mutually exclusive. Several may be in play. To maximize the impact on nervousness and perceived pressure of these implied threats, your best move here is to keep things vague.
Disagree on “done”
”Done” is about completion. “Off my plate” is about avoidance.
Some possible meanings of the word “done”:
- Our customers have had it for weeks and haven’t noticed anything needing changing
- Our customers just got it this second
- Our customers could get it if they wanted
- Our QA team is testing it
- The code is feature-complete on a branch
- It’s out of our hands
- Nobody can blame us
Time wasted shuttling between various kinds of “done” is key. Avoid disambiguating. Use “done” to mean whatever’s convenient for it to mean right now.
Disagree on scope
Preconceive your own notions of what needs doing. Again, avoid taking input from your customer and your development team. Also, avoid letting them take input from each other. If they talk to each other regularly, you’ve got two problems:
- They might more directly understand each other’s needs
- They might find creative ways to meet their needs in less time
It’s mainly (2) you’re worried about here, for which (1) will put you at risk if you let it. Don’t let it. Get shallow agreement on a list of work items and don’t let anyone make you deviate from it.
Don’t have a customer (sort of)
If there’s definitively no customer, you’ve got big problems. Since nobody’ll be affected, there’s no harm in shipping whatever you’ve got right now.
Let me restate: With no customer, there’s nothing standing between you and deployment.
This constitutes an enormous impediment to your goal. What you need instead is a feeling of certainty that your customers exist, plus a lack of connection to any of them. Now you know there’s someone out there who’s supposed to be able to judge whether today’s product is better or worse for them than last week’s, and who can remind you every now and then why what you’re doing matters. Furthermore, your team knows that they can’t get direct answers to the most fundamental questions about their work, and that they can’t determine whether they’re doing anyone any good.
Well done! You’ve severed the most important feedback loop in their work.
If you’ve got customers and you talk to them — or, worse, the team is talking to them — all is not lost. You’ve got lots of options for making it difficult to get new code deployed:
- Name an environment “production” without discussing uptime, reliability, or other aspects of service
- Limit who has access to “production” systems (this also helps later, to complexicate debugging!)
- Require several signoffs for each deployment
- Provide as little information as possible to those responsible for signing off, so they waver, sometimes ask for more information, and occasionally exercise their veto powers
- Make it just as difficult to roll back, which helps justify the elaborate signoff procedure
Now you’re considerably more likely to miss your deadline. If the team somehow manages to produce exactly what customers need, it’s going to take a while before they can get it. And otherwise, look how long it’ll take before customers can identify new needs, before the team can possibly hear about them, and before any work to address these needs can even begin!
Separate QA from Development
As part of your overall strategy, you need “QA” to be its own team and problem-identifying to be its own phase.
You want “Dev” working on their own, assuming QA will catch whatever they missed. You want QA working on their own, assuming Dev is following the most expedient path to claiming they’re done.
You want even the best-intentioned developer to say “done” as soon as possible and feel free of responsibility when it turns out not to be deployable. You want even the best-intentioned QA member to say “good enough” as soon as possible while finding a few unimportant bugs. You want Dev to be annoyed with QA when they find important problems, and you want QA to be annoyed with Dev when they repeatedly let important problems through.
This part of your strategy is very important. You slow down the work today by splitting it up, you slow down the work indefinitely by incentivizing placing blame over delivering value, and you make it very difficult to make predictable progress against the plan. At any given moment, it might look like you’re on target for a deadline when you’re really not. Perfect!
Separate Development from decision-making
If you’ve managed to keep your team from talking to the customer, good: they don’t know what you know about the business situation, so nobody but you is in a position to decide how the team should do its work.
You’ve heard Test-Driven Development takes longer because developers are also writing tests, and you’ve heard Pair Programming basically takes twice as long because two people are doing one person’s work, but you have the sneaking suspicion that pairing and TDD would save time by producing work output that’s frequently deployment-ready to begin with.
Can’t have that. What would happen to your standalone QA phase? What would happen if people’s attention were suddenly focused on your lengthy deployment signoff process?
Don’t let them pair or test-drive; above all, don’t let them make these choices themselves. Not on your watch. You’ve got deadlines to miss.
Keep code brittle
the only time we don’t change code is when there’s no existing code, a starting state that lasts about 10 minutes.
—Michael D. “GeePaw” Hill
As a codebase grows, it tends to get harder to change. Because this stems from the number of components in interaction, the difficulty grows much faster than, say, lines of code. Great news! The odds are already stacked in your favor.
You still need to be careful. Besides Pair Programming and Test-Driven Development, you need to keep an eye out for the other Extreme Programming technical disciplines. Beware Refactoring, Continuous Integration, Collective Code Ownership, and the like. They tend to have the effect of greatly reducing the risk of changing code. That’s why the decisions to use these practices belong with you, so you can veto them. You need code-change risk as uncontrolled as possible.
Scope creep of this article
I said this article would only be about missing software deadlines. It seems like it could pretty easily be about a lot more. Also, if you’re a developer, I could give you lots more specific ideas for making deadlines whoosh by. But I’ve already missed my self-imposed Wednesday deadline for this week’s blog post. Like I said, I’m an expert on this topic.
This list is far from exhaustive, but I hope — from the bottom of my backlog — that I’ve helped you get started on more reliably missing your deadlines.
Similar, but different
Creating my podcast feels like debugging. After quite a bit of trial and error, not knowing how far I am from identifying and solving the problem, eventually and suddenly it’ll be working for me. It takes however long it takes, and then it’s a source-control commit message away from being done.
In the case of the podcast, there’s more to it than that. Besides recording — the fun part — there are a bunch of manual steps to post-process, encode, publish, and publicize. I’m annoyed by repetitive steps in general, and by these in particular, because they stand between me and a hard-earned adrenaline hit. That makes me slightly less happy to have finished another episode. Being able to push a figurative button (or run a literal program) would make me slightly more happy instead. I want that, and I’m getting closer.
I do recognize I’m optimizing away from the bottleneck, considering the script takes me hours to write. I’m not proud of that. If there were a faster, easier way to reach into my brain and pull out something of equal value, I’d do it in a second, or however many seconds it’d take. Meanwhile, I observe that, thus far, incremental value correlates roughly with incremental effort. As a lazy person I very much want to reduce effort, but as a perfectionist I very much don’t want to compromise value.
Or so it seemed, until I got to experiment with another way. Pillar has been making a series of short (3 minutes or less, usually) videos on various topics pertaining to our business, and our artist/producer Darrin Hoover came up to our Michigan office this week with his multimedia rig and a bagful of black crayons. I talked into the microphone on a couple technical topics I feel strongly about, Darrin drew some images directly or imaginatively related to what I said, we took some footage of me manipulating the drawings on or around my face, and that was that. What a change of pace! — the interactive collaboration with a creative partner who reflected and amplified my ideas, the video aspect, the being blessedly uninvolved in the production process.
Compared to my usual, this was similar, yet different. I came away reinvigorated about my usual. (I’ll share the two videos as soon as they’re up.)
Rate of flow
My podcast production has had no buffer (with one exception). Every week, if I’m to publish a new episode, that requires that I bring a new episode into existence. This needs to change, effective immediately, because the travel routine that has afforded my writing routine will come to an end in mid-December, to be followed by six weeks of deliberately routine-busting time off. So if I’m to continue shipping new episodes on schedule through the end of January, then for the next five weeks I need to be writing more than two new episodes per week.
That’s never felt possible before, but it feels possible now. For one thing, I’ve seized on a new sequence of topics that excites me (including this week’s Refactor). For another, if I’m smart, I’ll learn from Darrin’s fast, unfiltered creative process and tune my design for bringing you 3 valuable minutes.
What you’re looking at here is a razor-thin allegory about incremental improvement — why it can be hard to start, and why once started it can be easy to continue — in the guise of a story about improvements to this website.
Now that I’m making more and better use of my new, simplified setup,
I’m motivated to seek further simplification. Unlike many other
this web page was neither brought into existence
when you asked for it nor recalled from an application-level cache
because someone else recently did so. This page already existed as
a plain old file on disk before you asked, and it’ll continue to
exist long after. It’s served slightly faster when it’s in the OS
kernel’s filesystem cache, but it’s served plenty fast to begin
with, because bringing it to you requires no application code to
run — just webserver code. That’s why my next major improvement
target is to stop needing Apache, a big featureful configurable
complicated webserver. It’d be neat if the small, just-featureful-enough,
and fairly simple
bundled with NetBSD (my server OS of choice)
sufficed to serve this site.
That’s a goal I couldn’t have seen from my previous traditional-CMS
vantage point, and it’s still
not a thing I can just go do. I feel pretty strongly about preserving
URLs that have ever been published; if not for the featurefulness
and configurability of Apache’s
mod_rewrite, many of the URLs
served under the old regime would have become broken under the new
one. In order to convince myself I no longer need this handful of
powerful URL-rewriting rules, I’ll need to convince myself the
permanent HTTP redirects have had an impact by observing that
old-regime links are hardly ever getting hit. And if that’s not the
case, I’ll need to convince enough of the referrers with those old
links to update them. Then, and only then, will switching webservers
finally be within my incremental reach.
Another new-regime payoff I hoped for was that because making small
improvements would be easier, I’d make more of them. That’s going
well, too. Textpattern had generated imperfect URL slugs for some
old posts, and that had always bothered me, but I’d never done
anything about it. Last week I finally fixed them, committed the
changes to source control, and added simple HTTP redirects (nothing
bozohttpd can’t do) for the old URLs. While I was at it, since I
discovered that “completionist” is more common usage than “completist”,
I did the same for my recent post on reading Twitter like a
lets me make changes across my entire site with familiar Unix tools
git diff and
git revert — rather than nerve-wracking,
potentially-destructive-and-hard-to-restore-from database operations.
Since I feel safer about making changes, I act on a higher percentage
of the little ideas that come to mind. On Monday, for instance, it
occurred to me that since my posts have tags, and there are already
autogenerated pages listing every post for each tag, I’d like to
add a page with a cloud-like list of all the tags. With ikiwiki,
it was really, really easy, so I did it. And once I’d done
it and was inspecting the result, I noticed that some old posts
needed more tags added. (Textpattern had no native tags, only
categories, no more than two per post.) And since this is Unix, I
grep for files containing the keywords I wanted to tag, or
find all the posts from 2014, pass them to my text editor for
quick fixing, and
git diff to double-check before publishing. So
I followed the chain of affordances and did all these things.
Exactly as I would not have bothered to in Textpattern.
Getting to where desirable improvements are available to be made in small increments can be difficult. It took me a couple weeks of disciplined effort to import my Textpattern site into ikiwiki in a way that I fully trusted. But once I got there, and improvements became easy to make and safe to try, it’s so rewarding to try them that I keep doing it.
One of these moons, before I know it and without making a concerted
effort, flipping the switch to
bozohttpd will be the next increment
in front of me. And then, because I can just do it, I’ll just do
Monday morning, in my regularly scheduled stupid-early cab to the airport, I reflected on where I’ve come from (besides being home, in bed, asleep, like a normal person) and where I’m going:
6 years ago today, I came back to software development. To relearn old tricks, uncover new ones, and enjoy. Headed to #agiledc to celebrate.
My celebration got off to a proper start when I didn’t miss James Grenning’s good-naturedly trenchant keynote, during which my still sleepy brain perked up at the mention of my colleague Erik Dietrich, in particular his How Developers Stop Learning: Rise of the Expert Beginner. For one thing, it induced a beperkifying spike of writerly ambition. Maybe, if I keep publishing something every week, I’ll eventually learn how to have written something worth being referred to by a respected keynoter. (Erik landed one of the guys who wrote the Agile Manifesto. Achievement unlocked.)
For another thing, hearing a familiar name put me in the right frame of mind to enjoy AgileDC in what is, for me, a whole new way of enjoying conferences. When I was an introverted outsider knocking on the door to this small, tight-knit community of human-loving, business-advancing Agilists, the primary way I felt I could extract value at conferences was to attend talks and workshops of interest. Now that I’m working in the field and getting to know some of the wonderfully generous and skilled folks in it, I spent most of my day in the “hallway track” I’d heard so much about. It was great. We conferred. I heard some terrific stories of personal growth interspersed with perfectly obnoxious jokes. And I saw some people I’d seen just last week.
Several of us June 2015 PSL participants schemed to hold a mini-reunion at the Toronto Agile Conference. Some were based in Toronto, and others got their talks accepted, so I accepted the need to get myself there from Detroit. I drove up Monday afternevening and joined a bunch of the already-fed speakers at a bar on a high floor overlooking downtown. (Toronto is for real. I want to visit again for real.) Almost everyone who was there I’d heard of, maybe Twinteracted with, but never met in person. I wasn’t a speaker; there was no need for them to have invited me along; and once again I found myself feeling, as I’ve been feeling so often lately, the generosity and inclusiveness of this community I’m getting to know.
Some of the folks I met in Toronto and D.C. have been invited to present at a new Agile Alliance Technical Conference being held in April 2016. What’s a little mind-boggling, considering what little I was doing a year ago, is that I’ve been invited too. And I’ll also be reviewing submissions for the Agile 2016 Leadership track.
There’s no way I can take in the pace of what’s happening for me and not see the pattern of generosity and inclusiveness. There’s no way I can not feel immense gratitude for all the people who’ve helped me find my new path and make my way along it.
I see it. I feel it. Thank you.