Skip to content
CS Capstone Handbook

AI Project Setup

Your repository exists, and your teammates have started pointing coding agents at it. Each agent knows only what the repository tells it, so one adds a dependency nobody agreed to and another reports a task finished that nobody can check. This page covers what you commit so every agent works from the same rules and can tell when its work is done. It doesn’t cover how to prompt and review, the CI (continuous integration) workflow, or the branch settings.

The Repository Is the Agent’s Onboarding

Section titled “The Repository Is the Agent’s Onboarding”

A coding agent starts every session with the code, the files you committed, and the task, and nothing else. A decision made in a meeting or a constraint the partner mentioned in an email is invisible to it until someone writes it into the repository. OpenAI’s Codex team makes the same point and treats a versioned docs/ directory as the system of record.

Birgitta Böckeler calls everything around the model its harness and sorts the parts two ways. Some steer the agent before it acts (she calls these guides); others check the result so it can correct itself (sensors). Each is either computational (a script: fast, and the same answer every time) or inferential (another model judging: slower and less predictable):

Steer before Check after
Computational formatter config, permission lists tests, type checker, linter, CI
Inferential AGENTS.md and the docs it points to a review pass in a fresh context

Use the computational row wherever a script can make the call, and keep the inferential row for what needs judgment. Strong types and clear module boundaries make computational checks cheap, which is one more reason your technical design matters.

What Goes in AGENTS.md

Section titled “What Goes in AGENTS.md”

AGENTS.md is an open format for agent instructions: a Markdown file at the repository root that the agent loads at the start of every session. Most coding agents read it, and a tool with its own filename can point that file at it, so write the content once rather than keep two copies that drift apart.

Put in it only what an agent can’t find out for itself, because every line costs context on every task. Gloaguen et al. (2026) (preprint) found that agents followed context files, but the files didn’t significantly change how often they solved a task, raised its cost, and didn’t help as repository overviews. dos Santos et al. (2026) (preprint) confirmed 58 of 100 popular repositories’ files restating rules a linter or formatter already checks. That leaves a short list:

  • The commands to install, build, test, and run, exactly as typed.
  • Conventions the code doesn’t make obvious.
  • Constraints, with a pointer to where the reasons live, such as docs/adr/.
  • Where the work is tracked and what done means (next section).
  • Boundaries in three tiers: always, ask first, never.

The tiers come from GitHub’s guidance on writing agent files. Ask first doubles as the stopping rule: keep working unless a decision needs a person or the next action is on that list. Write the never tier even if you write nothing else, because it’s the part that prevents damage, and put security in it, because that’s the boundary these files mostly leave out: Chatlatanagulchai et al. (2026) found security instructions in only 14.8% of the context files they classified, against 75.9% for testing.

For length, use a test rather than a number. Anthropic’s best practices ask of each line whether removing it would cause a mistake, and cut it if not. Kept to that test, the file fits on one screen:

AGENTS.md
## Project
Shift board for <partner>. Requirements: docs/requirements.md.
Architecture: docs/design.md. Decisions and reasons: docs/adr/.
Open risks: docs/risks.md; say so if a task touches one, don't edit it.


## Commands
npm ci, npm run dev, npm test (Vitest), npm run e2e (Playwright).
npm run check runs types, lint, and tests.


## Work
GitHub Issues; docs/agents/issue-tracker.md says how to read one,
what the labels mean, and how a PR links back. Done means the
issue's acceptance criteria and the Definition of Done in
CONTRIBUTING.md.


## Conventions
- All data access goes through src/server/repos/. No SQL in routes.


## Boundaries
- Always: run npm run check before calling a task done.
- Ask first: adding a dependency; changing the schema, auth, or
  deployment config.
- Never: commit to main, edit a test to make it pass, or paste
  partner data or .env contents into any tool.

The last line is your confidentiality rule written for the tools; the version for people belongs in the working agreement.

Some tools also keep a memory, but memory never shows up in a diff, so nobody reviews it: keep it for personal conveniences and put team rules in the file.

Define Done, Then Let the Agent Check It

Section titled “Define Done, Then Let the Agent Check It”

An agent stops when the work looks done to it. With no check to run, as Anthropic’s best practices put it, “you become the verification loop: every mistake waits for you to notice it.” So say what done is before it starts, at three levels:

  • The task names its finish line. Addy Osmani’s example: every payment endpoint is migrated to the new library, the old one is deleted, and the test suite passes.
  • The issue carries acceptance criteria someone could mark passed or failed.
  • The project has a Definition of Done that applies to every task, which AGENTS.md points to rather than each request repeating.

The criteria live in your issue tracker, which sits outside the repository, so the repository carries the map to it. A short document such as docs/agents/issue-tracker.md says how to read an issue, which label marks work ready for an agent, how a pull request links back, and when to file a new issue instead of fixing something in passing. Then “work on the next ready issue” is a complete instruction. GitHub Issues is one example; any board the agent can read works, and if it can’t reach yours, paste the criteria into the task.

Criteria become evidence only when the agent can run the checks behind them: tests, a type checker, a linter, and one command that brings up a working environment, all runnable without you, with error messages that say how to fix the problem. Ask for the test output, not the claim. What to test is a separate subject.

Don’t let the session that did the work be its only reviewer. A reviewer in a fresh context sees only the diff and the criteria, so it doesn’t share the author’s blind spots. Tell it to report only gaps against correctness and the criteria, because a reviewer asked for problems will find some whether or not they matter.

Hooks and Gates

Section titled “Hooks and Gates”

Everything in AGENTS.md is a request. Anthropic’s documentation says so: instruction files are advisory, hooks are deterministic. A hook is a script the agent’s tool runs on an event, such as before a command or after an edit, whether or not the model paid attention to your instructions. It can refuse a command, format every edited file, or keep the agent from stopping until the tests pass.

Start with the dull checks: no commits to the default branch, no edits to generated files, format after every edit. They block nothing ordinary, which keeps the team from switching them off. Where a list of allowed and denied commands will do, use your tool’s permission list instead of a script.

An agent hook is one of three layers, and each catches what the one before it misses:

Layer Runs Bypassed by
Agent hook When the agent calls a tool Working outside that agent
Git hook (lefthook, husky) On commit and push git commit --no-verify
CI and branch protection On the pull request Anyone on the bypass list

Anyone can skip a git hook, so the same checks run in CI, where branch protection turns a red build into a blocked merge. The workflow and the repository settings, including who can bypass them, are in those guides. A hook answers in milliseconds but stops one agent; CI takes minutes but covers every change. Put fast feedback in the first, and the check you depend on in the last.

Tools, Data, and Credentials

Section titled “Tools, Data, and Credentials”

The Model Context Protocol (MCP) is an open standard for connecting an agent to outside services. A server exposes tools the agent can call, such as fetching current library documentation, driving a browser, or reading a database schema. Add one when it answers a question the agent otherwise guesses at, not by default: Jain et al. (2025) found retrieved documentation improved generated calls to rarely used APIs but, with weak retrieval, hurt calls to common ones.

Treat every server as code running with your credentials. The MCP specification states that tools represent arbitrary code execution and that a server’s descriptions of its own tools are untrusted unless the server is. Simon Willison argues that the dangerous combination is private data, content an attacker controls, and a way to send data out: then a web page or an issue comment can instruct the agent to leak what it reads. Keep the three apart: give a database server read-only access to a development database, keep credentials in environment variables, and keep partner data out of every tool.

Skills

Section titled “Skills”

A skill is a packaged set of instructions, optionally with scripts, that the agent loads when a task calls for it; until then only its name and description take up context. The format is an open standard, the Agent Skills specification. Good candidates are jobs done every week and slightly differently each time, such as a review against the team’s conventions or working an issue through to a pull request. Read a published collection, such as Jesse Vincent’s Superpowers or Matt Pocock’s skills, before writing your own, and read any third-party skill before installing it, because it can bundle scripts the agent runs.

A skill and a hook can guard the same mistake, but a skill runs when someone invokes it or the agent judges it relevant, and a hook runs every time. Put a check you’d be sorry to forget in a hook.

Best Practices for AI Project Setup

Section titled “Best Practices for AI Project Setup”
  • Keep one AGENTS.md, and commit the decisions it points to under docs/.
  • Define done before the agent starts, and make every criterion something a check can confirm.
  • Enforce what you explain: AGENTS.md says why a boundary exists, and a hook or CI makes it hold.
  • Commit instructions, hooks, skills, and tool configuration, so every clone behaves the same.
  • Add a line when the agent repeats a mistake, and delete lines that no longer hold, because these files grow by small additions (Chatlatanagulchai et al., 2026) and a quarter of them are never touched after they’re created (dos Santos et al., 2026) (preprint).
  • Trip every check once on purpose, because a check with a typo in its path fails silently.

Some Truths About AI Project Setup

Section titled “Some Truths About AI Project Setup”

A team can run agents on a bare repository and ship; the cost arrives later and lands on whoever reviews. Denisov-Blanch et al. (2026) (preprint) found that among repositories whose first AI tool was an agent, those with no committed AI configuration took on roughly twice the rise in code complexity of those with one, though they caution that teams who write these files may be more disciplined to begin with. And Xu et al. (2025) (preprint) found that after Copilot arrived in Microsoft’s open-source repositories, core developers reviewed 6.5% more pull requests while their own code output fell 19%.

The evidence that instruction files help is thin and points two ways. Gloaguen et al. (2026) (preprint) found no significant change in how often agents solved tasks, at higher cost; Lulla et al. (2026) (preprint) found agents finished faster and used fewer output tokens with the file present, but didn’t measure whether the work was correct. Neither shows the file making an agent more correct; that comes from the checks.

A check people resent gets switched off, and one nobody has tripped may already be off. Git hooks have --no-verify, agent tools have a setting that disables hooks, and a hook that crashes or points at a mistyped path often lets the action through without a word. The team keeps delegating as though the check were there.

A weak criterion gets met literally. “The tests pass” is satisfied by deleting the failing test as surely as by fixing the code, and nothing in that finish line tells the agent to prefer the fix. That’s why the example’s never tier forbids editing a test to make it pass.

AI Project Setup in Industry and Academia

Section titled “AI Project Setup in Industry and Academia”

The conventions are consolidating in public. OpenAI released AGENTS.md and Anthropic released MCP, and both went to the Linux Foundation’s Agentic AI Foundation as a neutral home.

The teams building these tools hold their own agents to the same practices. GitHub’s cloud agent can’t approve or merge its own pull requests, per its risks and mitigations page, and OpenAI’s Codex team keeps its AGENTS.md to roughly 100 lines, used as a map into docs/.

In academia the evidence is young: Gloaguen et al. (2026) (preprint) tested Python projects only and Lulla et al. (2026) (preprint) a single agent, so treat their numbers as a first measurement.

References

Section titled “References”
  • Gloaguen, T., Mündler, N., Müller, M., Raychev, V., and Vechev, M. (2026). Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents? arXiv. Preprint. https://arxiv.org/abs/2602.11988
  • dos Santos, H. V. F., Costa, V., Montandon, J. E., Silva, L. L., and Valente, M. T. (2026). Configuration Smells in AGENTS.md Files: Common Mistakes in Configuring Coding Agents. arXiv. Preprint. https://arxiv.org/abs/2606.15828
  • Chatlatanagulchai, W., Li, H., Kashiwa, Y., Reid, B., Thonglek, K., Leelaprute, P., Rungsawang, A., Manaskasemsak, B., Adams, B., Hassan, A. E., and Iida, H. (2026). Agent READMEs: An Empirical Study of Context Files for Agentic Coding. ACM Transactions on Software Engineering and Methodology. https://doi.org/10.1145/3840295
  • Jain, N., Kwiatkowski, R., Ray, B., Ramanathan, M. K., and Kumar, V. (2025). On Mitigating Code LLM Hallucinations with API Documentation. IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). https://doi.org/10.1109/ICSE-SEIP66354.2025.00027
  • Denisov-Blanch, Y., Agarwal, S., Azaletskiy, P., He, H., Schaeffer, R., Miranda, B., Vasilescu, B., and Koyejo, S. (2026). A Few Pages of Markdown: Committed AI Configuration and Lower Quality Cost after Coding-Agent Adoption. arXiv. Preprint. https://arxiv.org/abs/2608.25241
  • Xu, F., Medappa, P. K., Tunc, M. M., Vroegindeweij, M., and Fransoo, J. C. (2025). AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden. arXiv. Preprint. https://arxiv.org/abs/2510.10165
  • Lulla, J. L., Mohsenimofidi, S., Galster, M., Zhang, J. M., Baltes, S., and Treude, C. (2026). On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents. arXiv. Preprint. https://arxiv.org/abs/2601.20404

Additional Readings

Section titled “Additional Readings”

Sources and further reading

Activities that exercise this