Shipping

Read this in fall week one, before you pick your spring rung; it gives you each outcome type’s path up the outcome ladder and the clocks to start now.

Your spring grade’s largest partner-evaluation facet, Verification and Validation, is scored against outcomes: merged upstream, in production, reproducible, in real use. Every project is scored on the same outcome ladder, who ran the software, from nobody outside the team to people who depend on it, and the top rungs are hard on purpose. The bottom two rungs read the same for every outcome type: your partner watched it run, or nobody outside the team ran it. Everything above them depends on the outcome type, and that is what the sections below map out.

Here is the fact that should reorganize your year: with AI tools, building is rarely the long pole anymore. The external world is. Maintainer review latency, app store review, IRB approval and data access, partner IT security review, recruiting users you don’t know: every one of these clocks is measured in weeks, none of them is under your control, and none of them can be compressed by working harder in spring week 8. The only move that works is starting the clocks early. That means planning backward from the spring rung you want, and starting in fall week one.

This guide is the playbook, one section per outcome type. Each section says what each rung means for that type (the partner evaluation page holds the ladder itself), walks the prerequisites rung by rung, gives a term-by-term timeline, names the common blockers, and lists the evidence you should be collecting for the partner survey. Your Definition of Shipped turns this playbook into a signed agreement; the Repo Checkpoint gates enforce the intermediate milestones.

FOSS

On the ladder: for FOSS the rungs read, from the bottom, a pull request submitted; reviewed and revised; merged; shipped in an upstream release. Note what separates submitted from merged: not your code, but a maintainer’s attention and approval. That is a resource you must cultivate, not a step you execute.

Rung-by-rung prerequisites:

Submitted: a working development setup for the upstream project, their contribution workflow understood (CLA or DCO signed, style and commit conventions followed, tests passing in their CI).
Reviewed: a maintainer chose to spend time on your PR. Review latency on large projects is commonly 2 to 8 weeks per round, and most substantive PRs take multiple rounds.
Merged: every review round addressed, rebased onto a moving main branch, and the change judged worth maintaining forever by people who will own it after you graduate.

The fall week 10 gate for FOSS teams is a first upstream pull request submitted. Aim past it: get a trivial PR merged by the end of fall. A typo fix, a docs clarification, a good first issue. Its purpose is not the diff; it is opening the maintainer channel, proving your toolchain end to end (fork, build, test, CLA, CI, review etiquette), and teaching you this project’s review culture while the stakes are near zero.

Communication norms matter as much as code. Read the project’s CONTRIBUTING file and recent merged PRs before writing anything. Discuss substantial changes in an issue or mailing list before building them; an unsolicited large PR is the most-rejected artifact in open source. Be responsive when review arrives: maintainers deprioritize contributors who take weeks to respond.

Term	What must be true
Fall	Upstream builds and tests locally for every member; CLA/DCO done; first PR submitted, ideally a trivial one merged; target issues discussed with maintainers
Winter	Substantive PR(s) submitted and in active review; at least one review round addressed
Spring	PRs merged; follow-up fixes handled; contribution noted in changelog or release notes if possible

Common blockers: review latency (weeks per round, worse around releases); CLA requiring employer or university sign-off; a maintainer disagreeing with your approach after you built it (discuss first); the project migrating or freezing mid-year; your patch conflicting with a moving main branch.

Evidence to collect: links to merged and in-review PRs, the review threads themselves (they prove engagement), issue discussions where maintainers endorsed your direction, changelog or release mentions, and any maintainer comments praising the work. Screenshot or archive these as they happen.

New Product or Game

On the ladder: for a new product or game the rungs read, from the bottom, friends tried a deployed build; a dozen strangers, or a completed user study; two dozen strangers; sustained use by tens of heavy or hundreds of light users. The jump from friends to a dozen strangers is the hard one: strangers cannot try software that is not deployed, and they will not arrive the week you need them.

Rung-by-rung prerequisites:

Friends tried it: a deployed URL or installable build. Nothing more, but nothing less: “clone the repo and run it” does not count as trying your product.
A dozen strangers: a production URL with a real domain and TLS, an onboarding flow a stranger survives without you on a call, a recruitment channel (subreddit, Discord, campus group, itch.io page), and telemetry so you can prove they came.
Two dozen strangers, and beyond: retention, which means fixing what the first dozen users hit, which means having had those users early enough to iterate.

Deploy the walking skeleton to a production URL by the end of fall. This is the fall week 10 gate. It flushes out the whole external dependency chain (domain registration, DNS, TLS, hosting account limits, billing) while there is nothing to lose. See the DevOps guide for how.

Know your store clocks if you are distributing through one. Apple: developer account approval can take days, and Apple reviews about 90% of submissions in under 24 hours, but that is the clock on one round. Rejections and resubmission loops are routine, first submissions get extra scrutiny, and review slows near holidays. Google Play: new personal accounts face a testing requirement (a closed test with a dozen testers over two weeks) before production access. For games, itch.io is the fast path: no review queue, upload and publish the same day. Ship on itch.io early even if a store launch is the end goal.

Add an analytics or telemetry starter in fall, not spring: a privacy-respecting page-view and event counter (Plausible, Umami, PostHog) or a simple event log. Your Definition of Shipped requires metrics; metrics require instrumentation that has been quietly collecting since before you needed it.

Term	What must be true
Fall	Production URL live (walking skeleton); domain, TLS, hosting sorted; analytics wired; store accounts created if relevant
Winter	Beta users onboarding themselves; feedback loop running; store test track passed if relevant; real-user session evidenced
Spring	Strangers using it; metrics dashboard showing it; store listing live or study executed

Common blockers: app store rejection loops, Play Console testing requirements, free-tier hosting limits hit at the worst moment, domain or account ownership stuck with one team member, and above all user recruitment: strangers take weeks to find, so the channel must exist by winter.

Evidence to collect: the production URL and uptime history, analytics screenshots with dates, user counts and testimonials, store listing links and review correspondence, user study protocol and results, and retention evidence: repeat-use counts or week-over-week active users, which is what separates the top rungs from a launch spike.

Research

On the ladder: for research the rungs read, from the bottom, a real-data run shown to your partner; your partner’s group ran it on real data and you fixed what they found; someone outside the team reran the pipeline and got your numbers; a stranger reproduced the result from the published artifact, which is what a paper is for. Underneath every rung sits an unstated prerequisite: data you are allowed to use and results someone else can reproduce.

Rung-by-rung prerequisites:

A pipeline first: it runs end to end on sample data, which is the fall week 10 gate.
A real-data run: the real dataset, which means data access agreements and possibly IRB approval, both of which are clocks you do not control.
Reran by others, then reproduced by a stranger: the reproducible artifact described below, a reproduced baseline to compare against, compute for larger runs, and enough calendar time to iterate after the first results disappoint you (they will).

Start the IRB and data-access clocks in fall week one. IRB review at OSU takes weeks even for exempt determinations, and longer if revisions are requested; data use agreements between institutions can take a full term. Your faculty partner knows these processes: ask in your first meeting what approvals the project needs and who files them. If human subjects are even plausibly involved (user studies count), ask explicitly.

Research teams have their own fall week 10 gate. Meeting it in fall means that when real data arrives, you are debugging science instead of plumbing.

Reproducibility is the bar. A result that only exists on one student’s laptop is a rumor. A reproducible artifact means: pinned dependencies (a lock file or container), versioned or precisely referenced data, seeded runs, one documented command from clean checkout to result table, and a README a stranger can follow. The test is literal: another team member, or your professor’s grad student, runs it fresh and gets your numbers. Reproduce the baseline before you attempt novelty: if you cannot reproduce the published number you are comparing against, you cannot claim to beat it.

Term	What must be true
Fall	IRB/data-access filings submitted; pipeline runs end to end on sample data; baseline identified and reproduction started
Winter	Real data in hand; baseline reproduced; first real experiments run; results tracked and versioned
Spring	Novel experiments complete; artifact reproducible by a stranger; paper or report drafted with your partner

Common blockers: IRB revision cycles, data agreements stalled in legal review, the baseline paper’s code not actually running, compute quotas, and the classic: results that arrive in week 8 of spring with no time left to interpret them.

Evidence to collect: approval letters and agreement dates, the reproducible artifact itself (repo tag plus instructions), baseline reproduction numbers next to published ones, experiment logs, your partner’s written confirmation that results are novel or submission-worthy, and the paper draft or preprint link.

Consultancy

On the ladder: for consultancy the rungs read, from the bottom, the partner tried a prototype; their feedback addressed in a newer version; the partner integrating it into production; the partner running it in operations. Read the top rungs carefully: they describe actions the partner takes, not you. Your job is to make those actions easy, safe, and early.

Rung-by-rung prerequisites:

Feedback on a prototype, then addressed: something deployed where the partner can click it, early enough that their feedback can shape the next version.
Partner integrating: the system running in the partner’s environment, which means their IT department has said yes: accounts provisioned, security review passed, data handling approved.
In production: all of the above plus a handoff the partner’s people can operate without you.

Ask about IT and security review in your first fall meeting. “Who has to approve software running in your environment, and how long does that take?” At small nonprofits the answer may be “the director, tomorrow.” At hospitals, school districts, government agencies, and large companies, the answer is a formal review measured in months, sometimes with penetration tests, data protection assessments, or vendor onboarding forms. Whatever the answer, put it in your Definition of Shipped with an owner and a start-by date.

Agree on acceptance criteria early and in writing. The “feedback addressed” rung is only checkable against criteria the partner endorsed before you built. Vague scope is how consultancy projects hit spring with a partner who feels the system “isn’t what we asked for.”

Stage the deployment: your own hosting first (fast iteration, demos), then a staging instance in the partner’s environment in winter, then production in spring. The staging step is the one teams skip and regret; it is where authentication schemes, network policies, and data access rules surface.

Finally, adopt the handoff-is-the-product mindset. You graduate; the system stays. Runbooks, admin documentation, credential transfer, a maintenance guide, and a trained partner contact are not paperwork after the real work: for the partner, they largely are the product, and they are what the spring evaluation remembers.

Term	What must be true
Fall	IT/security review process mapped with dates; acceptance criteria written and partner-endorsed; prototype demoed from your hosting
Winter	Security review in progress or passed; staging instance in the partner’s environment; feedback rounds addressed against the criteria
Spring	Production deployment in partner environment; handoff package delivered; partner operating the system without you

Common blockers: security review dead time (weeks with no visible progress), a single overloaded partner IT contact, credential and access provisioning, data that cannot leave the partner’s network (design for this early), and partner staff turnover mid-year.

Evidence to collect: the signed acceptance criteria, meeting notes and emails showing feedback rounds and sign-offs, security review correspondence, screenshots of the system running in the partner’s environment, the handoff package, and the partner’s written statement of production use.

Best Practices for Shipping

Start every external clock in fall: domain, TLS, hosting, store accounts, CLA, IRB, partner IT access. None of them runs faster for being started late.
Deploy the walking skeleton while there is nothing to lose. A production URL with a placeholder page flushes out the whole dependency chain at zero risk.
Instrument before you need the numbers. Analytics wired in fall is evidence in spring; analytics wired in spring week 8 is a screenshot of an empty chart.
Treat recruiting strangers as a workstream: an owner, a channel that exists, backlog items, starting in winter.
Put dates earlier than winter week 10 in your Definition of Shipped, and renegotiate the rung in the open when they slip rather than discovering the slip in spring week 10.
Read the ladder literally and plan for the rung you can prove, not the one you hope for.
Capture evidence the day it happens (the merged PR, the dashboard, the maintainer’s comment, the partner’s email) and link it from docs/shipped.md.

Some Truths About Shipping

The last 10% is not 10%. Teams that plan “build until spring week 8, ship in week 10” are describing a plan for the bottom of the ladder.
Every external clock starts later than you think and runs longer than they told you. The only defense is starting it before you need it.
A trivial thing shipped in fall beats an impressive thing shipped never. The walking skeleton, the typo PR, the sample-data pipeline: none of them is the point, and all of them are the point.
Strangers are not friends who haven’t heard of you yet. Recruiting users you don’t know is a project workstream with its own backlog items, not a tweet you send in week 9.
“The partner will deploy it after the course ends” is “feedback addressed” wearing “in operations” as a costume. If it is not in their environment by spring, it is not shipped.
Maintainers, reviewers, IRB coordinators, and IT departments are people with queues, not APIs with SLAs. Being early, polite, and responsive is a technical skill.
If your Definition of Shipped contains no date earlier than winter week 10, it is not a plan; it is a wish. Renegotiate the rung or move the dates.
Evidence collected in the moment costs minutes; evidence reconstructed in spring week 10 is the kind the Release and Metrics page cannot credit. Screenshot the milestone the day it happens.

Shipping in Industry and Academia

Industry spent two decades removing the ceremony between a commit and production. Continuous delivery, trunk-based development, and feature flags exist so that “shipped” is a daily event rather than a quarterly one, and companies such as Etsy and Amazon made deploying many times a day the norm well before it was fashionable. Google’s site reliability teams run a production readiness review before a service goes live, which is the top consultancy rung with a checklist attached. The pattern is the same at every scale: the thing that is hard is not building, it is the last mile into someone else’s environment.

Open-source maintainers judge contributors by merged pull requests, and the first merge is the hard one, which is why “good first issue” labels and onboarding programs exist. In research, the reproducibility crisis produced artifact evaluation: ACM conferences award badges for artifacts that are available, evaluated, and reproduced, and a paper with a reproduced artifact is worth more than one without. The top research rung is written with that bar in mind.

The common thread is that every field has learned, expensively, that working software nobody can run is not a result. The ladder is the course’s version of that lesson.