learn<\/em> the quickest seem the most likely to succeed over the long haul. This is not an original thought, for sure, but most companies don’t seem to have operationalized it formally.<\/p>\nMaybe shipping is actually just a subset of learning, or at the very least, in service of it. Put differently, shipping can be a great way to learn, but it’s not the only way, and it’s not always the best way. If all of that is true, how might we re-orient the development process around learning?<\/p>\n
A\u00a0provocative\u00a0approach\u00a0would be\u00a0to prioritize learning<\/em> as the most important regular deliverable<\/strong> of engineering\/product\/design teams. Let’s say you are a manager, executive, or anyone else interested in a particular product within a company. Here is the sort of progress report you are probably used to seeing from whoever is leading the team building the thing:<\/p>\nSEPTEMBER<\/strong><\/p>\n\n- Launched movie recommendations<\/li>\n
- Hired 2 engineers<\/li>\n
- Fixed 30 bugs<\/li>\n
- Went into beta with collaborative filtering<\/li>\n
- Drafted product plan for internationalization<\/li>\n<\/ul>\n<\/blockquote>\n
When written more with more structure, they can take the form of OKRs (Objectives & Key Results) or KPIs (Key Performance Indicators).<\/p>\n
While useful in showing that the team isn’t sitting on their hands, it doesn’t really hold anyone accountable for what good all of that is doing for the organization. What caused the team to launch movie recommendations and what will that mean for the business? What will 2 more engineers accomplish? What is the impact of the bugs fixed? What do we hope to learn from the collaborative filtering beta and how will we know we should even launch it? What have we learned about internationalization in terms of how much bigger it will make our business or how much of an effort it will be?<\/p>\n
A list like the one above is at best a tactical inventory, but at worst a veil that\u00a0masks how little actual learning may be occurring. In other words, it may make you look like you’re moving forward (good!) at the expense of ignoring how little of a difference you are making (bad!). What if, instead of delivering something like that every week, month, or quarter, you delivered something more like this:<\/p>\n
SEPTEMBER<\/strong><\/p>\n\n- What we learned:<\/strong> Removing steps from our onboarding experience did not reduce\u00a0user\u00a0confusion. Instead, clearing up language such that users felt they were making progress resulted in the greatest gains (link to more details here).<\/li>\n
- Cost to learn:<\/strong> One researcher, one prototyper, one week of calendar time, and $2000 in participant fees.<\/li>\n
- Plan to proceed:<\/strong> Productionize new language within one month.<\/li>\n
- What’s needed to proceed:<\/strong> Some internationalization help… we’ll outsource this.<\/li>\n<\/ul>\n
<\/p>\n
\n- What we learned:<\/strong>\u00a0We have entire tab in our app \u2014 Theaters \u2014 that only 1% of our users regularly use. Additionally, we have another feature \u2014 Friends \u2014 buried under a sub-menu, that almost half of our users use regularly. We aren’t sure what the effects would be if replace Theaters with Friends, but it seems like the right thing to do.<\/li>\n
- Cost to learn:<\/strong>\u00a0One PM, from one day of digging\u00a0through data.<\/li>\n
- Plan to proceed:<\/strong>\u00a0Run an experiment swapping in a Friends tab in place of\u00a0the Theaters tab and see what effect it has on metrics.<\/li>\n
- What’s needed to proceed:<\/strong>\u00a0We have no dedicated data scientist in our group. Would be great to have one, but short of that, if we could just get a day per week of someone’s time, we’d take that.<\/li>\n<\/ul>\n
<\/p>\n
\n- What we learned:<\/strong>\u00a0Our projects take at least 30% longer than they should because of an outdated build process that has too many dependencies. In addition to the cost in extra time spent, we have also lost engineers and designers due to this frustration.<\/li>\n
- Cost to learn:<\/strong>\u00a0We’ve been paying this cost in slow development and high\u00a0attrition for at least two years.<\/li>\n
- Plan to proceed:<\/strong> We just dedicated two weeks of an engineer’s time to investigate the issues and what it would take to fix. Our hope is to stack rank what’s easiest and most valuable to fix and then attack the list aggressively.<\/li>\n
- What’s needed to proceed:<\/strong>\u00a0We need air cover from executive leadership for probably about a month while we get this fixed. Meaning, we need to pause our current product work and not have to answer questions about why. This is important work, leverageable by the entire organization.<\/li>\n<\/ul>\n<\/blockquote>\n
What we learned<\/strong>\u00a0is really the key deliverable, and something potentially interesting to many people across the company. Cost to learn<\/strong>\u00a0is a chance to optimize for and highlight efficiency, as spending a year of engineering time carries much greater opportunity costs\u00a0than running some\u00a0efficient, qualitative\u00a0testing. Plan to proceed<\/strong>\u00a0lets people know the outcome of the learning and how it affects the way forward. And finally, what’s needed to proceed<\/strong>\u00a0spells out what a manager, executive, or anyone outside the core team can do to help.<\/p>\nThis is a rough sketch and could use plenty of iteration and refinement, but one thing I like about it is that while there are plenty of legitimate and illegitimate excuses for not shipping, there is really no excuse for not learning<\/em>. In other words, a team could spend an entire quarter (or much longer) shipping absolutely nothing, even though they are doing all the right things. However, if a team is not learning quickly and regularly, they are almost certainly doing something wrong.<\/p>\nI also like that a re-orientation around learning gets people thinking in an entirely different way.<\/p>\n
“What’s the minimum we can ship with?”<\/em> becomes “How might we learn X most quickly?”<\/em><\/p>\n“What do we want to release next quarter?”<\/em> becomes “What is most important for us to learn\u00a0about our product\u00a0next quarter?”<\/em><\/p>\n“How can we ship more often?”<\/em> becomes “How can we increase the pace of our own learning?”<\/em><\/p>\nNone of this means you won’t also have to ask questions about shipping, but shipping simply becomes another way to learn… alongside research, prototyping, and many other methods. It’s still what you ultimately want to get to, but it’s more a by-product of the methodical learning you hold yourself and all teams within the company accountable for every week, month, or quarter.<\/p>\n
Some of today’s\u00a0companies who learn and evolve the quickest have surely already operationalized some of this sort of thinking, even if informally or by accident. If you’re reading this and you’re thinking to yourself “we already do all of that!”, then that is fantastic (and I would love to hear more about your process in the comments)! If, however, this sort of re-orientation sounds refreshing to you, start talking about it with your team and see what steps you can take to try it out on a project or two.\u00a0I’m curious to hear what works best for everyone.<\/p>\n
(This post also available on Medium<\/a>.)<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"“What did you ship last quarter?” “When is this going to ship?” “Real artists ship.” The verb\u00a0“ship” has a long history in the software development world and before that, the physical world. In the physical world, it originally meant “to transport\u00a0something on a vessel”, and in the software world, it meant “to press a tape\/disk\/CD […]<\/p>\n","protected":false},"author":3,"featured_media":28362,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[41,37,36,282,39],"tags":[],"class_list":["post-28356","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-business","category-code","category-design","category-original","category-technology"],"_links":{"self":[{"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/posts\/28356","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/comments?post=28356"}],"version-history":[{"count":0,"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/posts\/28356\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/media\/28362"}],"wp:attachment":[{"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/media?parent=28356"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/categories?post=28356"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mikeindustries.com\/blog\/wp-json\/wp\/v2\/tags?post=28356"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}