Project Partner Evaluation
Canvas is the source of truth for this term’s assignments. This page may already show changes Canvas does not have yet.
25% of your grade every term is based on your interactions with your project partner, rated by them in two surveys that the instructors review before any score stands:
- The Midterm Pulse (5% of the term grade): four statements about how the team works with them, each with a comment box, answered in about five minutes.
- The End-of-Term Survey (20% of the term grade): the six facets below, each scored at 100%, 90%, 80%, 70%, or 50% of its points; spring Verification and Validation adds a 60% level.
If your team has no project partner, as on a student-proposed project, your mentor (sometimes an instructor) stands in as the project partner and rates the same six facets. This 25% is always scored and never redistributed, so every student in the course is graded on the same four blocks of 25%.
Fall and winter are progress-focused; spring scores outcomes. The facet weights move with the year, and each term’s rubric gives them as points: fall is won on requirements, winter on building, spring on the shipped outcome. If a team changes projects mid-year the weights do not change.
How the scoring works
Section titled “How the scoring works”Every facet has three written anchors in the rubric, worth all of the facet’s points, 80% of them, and half of them. A partner scores 90% or 70% when the team sits between two anchors. The spring Verification and Validation facet is scored on the outcome ladder, which has six levels of its own.
Under each facet is a short list headed What it looks like. It is the same list for both readers: for the student, it’s what to do every week; for the partner, it’s what to look for before choosing an anchor. In the lists, “you” is the partner. Nothing on the lists needs a rubric to be understood, and nothing on them is optional in a professional engagement. Discuss the lists with your partner in the first meeting of every term.
Under every rating the partner can leave a comment, and must leave one for any answer below the top answer, saying what would get the team there. On the spring outcome ladder the comment is optional. Your team reads these comments as written, with the score.
The survey also asks the partner to flag any specific student who did not contribute; that flag starts the individual review described under Instructor Review below. The flag, and anything else the partner writes to the instruction team, is not shared with your team.
If Your Partner Does Not Answer
Section titled “If Your Partner Does Not Answer”A survey your partner (or the stand-in scoring your team) does not answer by the time it closes scores exactly the lower bound of an A on the grading scale, for every student on the team. This applies to the Midterm Pulse and the End-of-Term Survey separately.
Midterm Pulse
Section titled “Midterm Pulse”Four statements, each answered on a five-point scale from Strongly disagree to Strongly agree, and each worth a quarter of the pulse; the pulse rubric below turns each answer into points. Each statement is drawn from the facet lists below, so the way to score well in week 6 is the way to score well in week 10.
| Item | Statement your partner rates | Drawn from |
|---|---|---|
| Responsiveness | The team answers my messages within one working day, with an answer or a date. | Communication |
| Professionalism | The team arrives at our meetings with a written agenda or status, and every student speaks. | Communication |
| Delivery quality | What the team has shown me so far, from written requirements to running software, matches what we agreed. | Requirements and Specifications; Design, Implementation, and Deployment |
| Reflection | The team asks for my feedback and acts on it, or tells me why not. | Reflection; Verification and Validation |
Reflection
Section titled “Reflection”Whether the team learns from each sprint and shows it.
What it looks like
- The team can say, without being asked, what the last sprint taught them and what they changed because of it.
- Feedback you gave appears in the next iteration, or the team tells you why not.
- Retrospective outcomes reach you as changed behavior (a new meeting rhythm, a different demo format).
- The team states the project’s impact and its intellectual-property terms in their own words.
Requirements and Specifications
Section titled “Requirements and Specifications”Whether the team has written down what it is building, in terms the partner agreed to.
What it looks like
- Goals are written down, stated so they can be measured, and you have agreed to them.
- The team asks before assuming; an unclear item comes back as a question within the week, not as a surprise in a demo.
- Priorities are explicit: the team can tell you what is in, what is out, and what is next, and you have seen the list.
- The written requirements change as understanding changes, and you can see what changed.
Goals that are not stated and measurable cannot be evaluated; that is the first conversation of fall.
Design, Implementation, and Deployment
Section titled “Design, Implementation, and Deployment”Whether working software exists, advances, and can be run by someone other than the team. Fall carries little weight here; the goal is to have the implementation and a deployment path under way so winter starts at speed.
What it looks like
- You can open the current version at any time (a link, a build, or a branch you can run), not only at demos.
- Milestones you were told about are met, or renegotiated before they slip.
- Design decisions that affect you (data, integrations, hosting, cost) are explained before they are made.
- Documentation exists for whoever runs this after the team is gone.
“Deployment” means whatever puts the work where it’s used: a hosted service, an installable build, contributions to an open-source project that include every necessary change, documentation included, and work in its codebase without regression, or code and artifacts documented well enough to be reproduced.
Verification and Validation
Section titled “Verification and Validation”Whether the team tests against a written approach the partner has seen, and collects and acts on feedback from the partner or users; in spring, who outside the team ran the software.
What it looks like
- Tests exist, and the team can show you the latest run.
- Feedback from you or from users is collected every sprint, and you can see where it went.
- The team states, in writing, what “verified” means for this project and against which expectations of yours.
- In spring, the team can name the level of the ladder it is on and the evidence for it.
Spring
Section titled “Spring”In spring the scale is one ladder for every project, the outcome ladder: who ran the software. Read it literally. Every level above the partner watching it run describes someone outside the team running your work, and the top level describes people who depend on it.
| Level | What it looks like |
|---|---|
| People outside the team depend on it (full points) 40 of 40 points | For example, the partner's staff run it in their daily operations; a release of the open-source project includes the team's contributions; someone who didn't build it, such as a graduate student in the lab or a peer, reproduced the team's result from the published artifact without the team's help; people outside the team keep using it on their own (tens of heavy or hundreds of light users). |
| Someone outside the team runs it in their own environment (90% of the points) 36 of 40 points | For example, the partner is integrating it into production; the open-source project merged the team's contributions; someone outside the team reran the pipeline on real data and got the team's numbers; two dozen people outside the team used it, not friends or classmates asked as a favor. |
| An outsider ran it and the team acted on what they found (80% of the points) 32 of 40 points | For example, the partner's feedback on a prototype is addressed in a newer version; maintainers reviewed a pull request and the team revised it; the partner's group ran it on real data and the team fixed what they found; a dozen people outside the team used it, not friends or classmates asked as a favor, or a user study ran to completion. |
| An outsider ran it once (70% of the points) 28 of 40 points | For example, the partner tried a prototype; a pull request is open in the open-source project; the partner ran it once on real data, on the team's setup; friends tried a deployed build. |
| The partner watched it run on their real inputs (60% of the points) 24 of 40 points | A live demo or a recording, but nobody outside the team has operated it. |
| Nobody outside the team has run it (half the points) 20 of 40 points |
Getting to a real user, a real deployment, or a real merge is the part that takes a year of lead time, and it’s what the Shipping guide maps out. Its sections are guidance; they do not change the scale.
Your Definition of Shipped does not replace the ladder. It makes your level concrete. Your Definition of Shipped says what your target level means for your project: which environment, which users, which open-source project, measured how. That is what your partner scores against in spring, and it’s why you write it in winter, while there is still time to act on it.
Custom scoring is for genuine deviation, not for restating the ladder. If your partner’s expectations differ from the ladder in kind rather than in detail, they can score on a custom scale from 50% to 100% of the facet’s points. The standing example is a project whose right outcome is zero users yet, such as a tool that must not reach patients or operators until a certification that won’t arrive before spring week 10; it is scored on the user activities it did run, such as supervised trials and reproduced results. Agree that in the Definition of Shipped in winter, not in spring week 10.
Teamwork
Section titled “Teamwork”Whether the whole team shows up, any member can answer for the project, and the work is visibly shared.
What it looks like
- The same people show up; an absence is announced to you in advance.
- Any team member can answer for the project, not only the one who did the work.
- Disagreements reach you as decisions the team made together, with the options it weighed.
- Work is visibly shared: over a term, the demos and the questions come from different people.
Students also evaluate each other directly every term, and the students’ end-of-spring peer survey uses the CATME teamwork dimensions. See Peer Evaluations.
Communication
Section titled “Communication”Whether the partner always knows where the project stands without asking.
What it looks like
- Every student speaks at every meeting.
- The team arrives with a written agenda or status, and sends notes with decisions and next steps within a day.
- Demos show working software, not slides about it.
- A question gets an answer or an “I will find out by X”, and X holds.
- Written updates arrive on the cadence you agreed, without being chased, and a message gets an answer within one working day.
- Documents you receive are short, current, and answer the question they were sent for.
Instructor Review
Section titled “Instructor Review”An instructor reads every response, to the Midterm Pulse and to the End-of-Term Survey, before its score stands. They look closer when, for example, a facet sits at the low anchor, a comment contradicts its rating, or what the instruction team sees of the team contradicts a rating. A review can move the team’s score in either direction.
On a survey your partner answered, an individual adjustment changes one student’s score rather than the team’s. It starts from a partner’s flag or from the instructors’ own review of the record: your peer evaluations, the contribution lines in your sprint notes, the repository (NDA teams: as shown at the walkthrough or the defense), and your individual work. It’s applied facet by facet (statement by statement on the Midterm Pulse), in either direction, as far as the record supports, down to zero. The reason you get cites the record, never a teammate’s comment or your partner’s private answers.
If you don’t contribute, you can lose points here and in the individual contribution half of each sprint note.
How Students Can Protect Themselves
Section titled “How Students Can Protect Themselves”The best protection is visible, traceable contribution throughout the term:
- Commit code to your own branches and open or close your own issues; don’t let a teammate commit your work on your behalf.
- Write your own contribution lines in sprint notes and your own individual assignments; a teammate who writes your contributions for you undermines the traceability that protects you.
- Make every contribution measurable and attributable to you. If your work isn’t easily traceable (e.g., design discussions, testing, documentation), make sure to document it in a way that can be corroborated (e.g., meeting notes, issue comments, pull request reviews).
- If you’re blocked, tell your team and your project partner early. If you raise a blocker and adapt, you are treated very differently from a student who goes quiet.
- Peer evaluations reflect your teammates’ real-time view of your work. If you’re having a difficult term, address it on your individual page of the term retrospective and raise it with the instructors early; waiting until grades are posted is too late.
What You Submit
Section titled “What You Submit”Nothing. Your project partner, or whoever stands in for them, completes both surveys.
Rubric
Section titled “Rubric”Midterm Pulse
Section titled “Midterm Pulse”The pulse has its own scale, separate from the six facets of the End-of-Term Survey.
| Criterion | Ratings |
|---|---|
| ResponsivenessThe team answers my messages within one working day, with an answer or a date.25 points | |
| ProfessionalismThe team arrives at our meetings with a written agenda or status, and every student speaks.25 points | |
| Delivery qualityWhat the team has shown me so far, from written requirements to running software, matches what we agreed.25 points | |
| ReflectionThe team asks for my feedback and acts on it, or tells me why not.25 points |
End-of-Term Survey
Section titled “End-of-Term Survey”| Criterion | Ratings |
|---|---|
| ReflectionWhether the team learns from each sprint and shows it.15 points | |
| Requirements and SpecificationsWhether the team has written down what it is building, in terms the partner agreed to.40 points | |
| Design, Implementation, and DeploymentWhether working software exists, advances, and can be run by someone other than the team.15 points | |
| Verification and ValidationWhether the team tests against a written approach the partner has seen, and collects and acts on feedback from the partner or users.5 points | |
| TeamworkWhether the whole team shows up, any member can answer for the project, and the work is visibly shared.15 points | |
| CommunicationWhether the partner always knows where the project stands without asking.10 points |
| Criterion | Ratings |
|---|---|
| ReflectionWhether the team learns from each sprint and shows it.15 points | |
| Requirements and SpecificationsWhether the team has written down what it is building, in terms the partner agreed to.15 points | |
| Design, Implementation, and DeploymentWhether working software exists, advances, and can be run by someone other than the team.40 points | |
| Verification and ValidationWhether the team tests against a written approach the partner has seen, and collects and acts on feedback from the partner or users.10 points | |
| TeamworkWhether the whole team shows up, any member can answer for the project, and the work is visibly shared.10 points | |
| CommunicationWhether the partner always knows where the project stands without asking.10 points |
| Criterion | Ratings |
|---|---|
| ReflectionWhether the team learns from each sprint and shows it.15 points | |
| Requirements and SpecificationsWhether the team has written down what it is building, in terms the partner agreed to.5 points | |
| Design, Implementation, and DeploymentWhether working software exists, advances, and can be run by someone other than the team.20 points | |
| Verification and ValidationWho ran the software, on one ladder for every project. Where the Definition of Shipped records a custom scale, the partner scores on it instead, from half to all of the points.40 points | |
| TeamworkWhether the whole team shows up, any member can answer for the project, and the work is visibly shared.10 points | |
| CommunicationWhether the partner always knows where the project stands without asking.10 points |
Activities That Prepare This
Section titled “Activities That Prepare This”- Regular Stand-Up Meetings: the Teamwork and Communication lists; the habit of every voice, every time.
- User Story Mapping: the Requirements list; written, prioritized goals your partner has seen.
- External Feedback Session: the Reflection and Verification lists; feedback collected and visibly acted on.
Browse Team and Workflow and Presenting when these run out.