Working with ChatGPT, Tested

A guide to OpenAI’s assistant for people who direct it rather than program: which ChatGPT you are using, how to give it rules it keeps, and which popular tips held up on a small folder of files

What works when you direct ChatGPT and Codex to do real work: the apps and plans, the AGENTS.md file it reads before it starts work, five habits each tested plainly and carefully on the same made-up folder, the tips that made no difference, and where your files go. Last checked November 2026.
Author

Behzad Samadi

Published

November 11, 2026

A paper-craft garden committee table: an open laptop with a terminal window, a ledger with one row struck through and a tally, a paper adding machine, a fountain pen, seed packets, a watering can, a mug, and the five-dot AI glyph above. Title: Working with ChatGPT, Tested.

Last checked 11 November 2026, on GPT-6.1 Sol, in Codex CLI 0.166.0.

This guide is for someone who uses ChatGPT to get things built or sorted out, not to write code themselves: a committee’s paperwork, a spreadsheet, a small app. It covers the assistant as a whole: the chat in a browser, the desktop app with its Chat, Work and Codex modes, and Codex on its own, the version that works on the files in a folder.

The practices in this guide were tried on the same small folder of made-up files: a garden club’s seven sets of committee minutes, with untidy names and one misleading date, and a members list with a fee column that has four traps in it. Each task was asked twice, once plainly and once with the habit being tested, three times each, in a fresh copy of the folder for each run. The counts are small, so the guide says “in three runs out of three”, not “always”. How the tests were set up is at the start of Practices that held up.

If you have not yet let an assistant work on a folder, start with Let It Open the Folder, which sets up the ChatGPT desktop app and explains why a folder beats pasting into a chat.

Other guides in this series: Working with Claude, Tested and Working with Grok, Tested.

NoteIn short
  1. ChatGPT is good at reading a pile of documents and getting the facts right. With a plain question or a careful brief, it found every open action, the right fee total and the missing figure.
  2. Do by hand: the final read for your house style, unless the rules are written down. Plain requests broke one to three of six simple writing rules. Nothing in these tests called for another assistant.
  3. The habit that made the most difference is the instruction file: a file called AGENTS.md in the folder. With the rules in it, every rule held in three runs out of three. Correcting it once and asking it to update that file worked as well as writing the file yourself.
  4. To protect your files, ask it to plan and to show its check before it changes anything, and keep connectors off. Asked, it stopped, showed every rename, and proved afterwards that no text was lost. Connectors and plugins were off in the tests; OpenAI notes that their traffic is not held by Codex’s own network controls.

Which one you are using

ChatGPT now comes as one app with three ways to work, plus Codex on its own. They share your account and, for Work and Codex, the same usage allowance.

What it is Works on your files
Chat chatgpt.com in a browser, the phone app, or the desktop app Files you upload
Work The same app, with Work turned on: it plans, uses tools and hands back a finished file Uploaded files; in the desktop app, a folder you open
Codex The coding agent: a mode in the desktop app, a command-line app, an editor extension, and a cloud version The folder you open or start it in

OpenAI’s own short version: “Use Chat for a question or back-and-forth. Turn on Work in the switcher when you want ChatGPT to carry a larger task through to a reviewable result. Select Codex when you want developer views or more technical detail, especially for software development.” Which plans include which, prices and limits are in The moving parts.

This guide was tested in Codex’s command-line app, because a script can run it the same way thirty times and keep what it wrote. The desktop app’s Codex mode uses the same settings file and reads the same instruction file, so what holds here should hold there; that was not tested. Chat and Work were not tested; where a practice depends on them, the guide gives OpenAI’s own page instead. For setting up Codex beside other coding agents, see One Repository, Three Agents.

Giving it context

ChatGPT starts each conversation knowing little about you. There are four ways to change that, and they are not the same thing.

Files and folders. In Chat you upload them. In the desktop app you open a folder, and ChatGPT “can read and change files in every attached folder”. Codex on the command line works on the folder you start it in.

Projects (Chat, Work and the desktop app). A project keeps “related chats, files, instructions, and sources together”, and its instructions apply to every chat in it. One point is easy to miss: a project on the web “doesn’t provide direct access to a folder on your computer”, so you upload what it should use. In the desktop app a local project is attached to folders instead. OpenAI’s advice: one chat per outcome, inside the project.

Custom instructions and memory (Chat and the desktop app). Custom instructions, under Settings > Personalization, are preferences “you want ChatGPT to follow across chats”, such as how long answers should be. Memory carries what it learns about you from one chat to the next. Two things to know. ChatGPT on the web and Codex keep separate memories: Codex’s are files on your own computer, off until you turn them on. And OpenAI says not to rely on memory for rules: “Treat memories as a helpful recall layer, not as the only source for rules that must always apply.”

The instruction file (Codex, in the desktop app or on the command line). A plain text file called AGENTS.md in the folder. “Codex reads AGENTS.md files before doing any work.” Write in it what you would otherwise repeat. Only that name is read, unless you configure others: a CLAUDE.md left by another assistant is not loaded as instructions. A personal one in your home folder (~/.codex/AGENTS.md) applies everywhere; OpenAI says that is where Codex keeps your custom instructions. OpenAI’s advice is to keep it short (“A short, accurate AGENTS.md is more useful than a long file full of vague rules”) and to add a rule only “after you notice repeated mistakes”.

What the instruction file did

The task: Write the notice to members about the work day, from the committee’s notes. Save it as notice.md. The club has six house rules for anything sent to members:

- Dates in full, with the weekday and the year: Saturday 14 November 2026.
- Times on the 24-hour clock: 09:30, not 9:30 am.
- British spelling.
- No exclamation marks.
- Under 120 words, heading and signature included.
- Signed "The committee", with no personal names.

Run three times without the rules, and three times with them saved as AGENTS.md in the folder. Marked out of 10: four facts (the date, the times, gloves, soup) and the six rules.

Run 1 Run 2 Run 3
No instruction file 8 9 7
Rules in AGENTS.md 10 10 10

Every notice had the facts right. Without the file, ChatGPT wrote “9:30 a.m.”, added an exclamation mark, or signed as “The Alder Lane Community Garden Committee”: one to three of these in each run. The run that scored 8 lost one of its two marks on signing “The Committee” with a capital C; the rule quotes “The committee”, so it was marked strictly. It already wrote the date in full, used British spelling and kept under 120 words without being told. With the file, every rule held, and the notices were shorter: 63 words each, against 82 to 102. Codex’s own session records showed the file loaded before it began, in all three.

Practices that held up

How the tests were run. On 11 November 2026, in Codex CLI 0.166.0. Model: gpt-6.1-sol asked for on the command line, and each run reported gpt-6.1-sol. Effort: not set, so the test account’s setting applied: low; no OpenAI page read says that is the default. Permission: codex exec, which does not stop to ask, with the sandbox set to workspace-write, so it could write only in the folder. Connectors and MCP servers were kept out with --disable plugins, --disable apps and each MCP server switched off on the command line; Codex’s memories were off too. Web search: the default cached search was on; no run used it. Instruction files: the tester’s own global instruction file loaded; it added no rules. Each prompt was run three times plainly and three times with the practice, each in a fresh copy of the folder.

Brief it, do not just ask it

Plain: What do we still need to do?

With the practice: Goal: a list of the committee’s open actions, so nothing is forgotten before the garden closes for winter. Read every file in notes/. An action is something a named person, or the committee, agreed to do. It is open unless a later note says it was done. If an action was carried over, list it once, with its latest date. Write actions.md with one table: action, who, due date, the note it was last mentioned in. Under the table, list anything you were unsure about and why. Do not change any other file.

Marks out of 9 Run 1 Run 2 Run 3
Plain 9 9 9
Brief 9 9 9

The plain question found all five open actions in three runs out of three, with the right people and dates, including the one that changed hands in September. It also mixed in things nobody had agreed to do (running the work day, chasing an unpaid fee, in one run handing it to someone the notes had not given it to). The brief gave exactly the five, in a file, and listed its doubts under the table: the 26 August date, who supplies the soup, who pays the roofer. The difference is in what you have to read and check afterwards, not in what it found. This is the shape OpenAI recommends too: “Goal”, “Context”, “Output”, “Boundaries”.

Plan first when it is going to change files

Ask for the plan, read it, then say go.

With the practice, first message: I want the notes folder tidied: every note named for the date of its meeting, as YYYY-MM-DD-committee-meeting.md, and a notes/index.md that lists each note with one line on what was decided. The text of the notes must not change. Do not change anything yet. Tell me your plan: what you will rename to what, how you will work out each date, and how you will check afterwards that no text was lost. Then stop.

Second message, same conversation: Go ahead with that plan. When you finish, tell me what you checked.

The plain version was Tidy up the notes folder: consistent file names, and an index. One file name is a trap: minutes 25 aug.md holds the meeting of 26 August, which was moved because of a storm.

Run 1 Run 2 Run 3 Said how it knew nothing was lost
Plain passed passed passed no, in all three
Plan first passed passed passed yes, in all three

A checking script confirmed each result: seven notes, text unchanged, each named for its own date, all listed in an index. The plain runs did the job well. All three caught the 26 August trap; they named the index notes/README.md. (The plain runs pass the test’s check for the plain prompt, which accepts any file name that starts with the right date and any name for the index. Its stricter check, which wants the names only the careful prompt gives, fails them on the file or index names alone.) What the plan added was what you see before anything moves: all three stopped and changed nothing, as asked, gave a table of old and new names, pointed out the 25/26 August mismatch, and said they would take a fingerprint (a SHA-256 checksum) of every file before and after. Afterwards they reported that every file matched. OpenAI’s own advice: “Ask for a plan when the approach matters.” Codex also has a /plan command, in the desktop app and in an interactive session; it was not tested here. The Grok guide found that its plan mode does what its own plan says, so read the plan before you say go.

Outside what was asked. All three plain runs also linked their index from the folder’s README.md, which nobody asked for. All six runs looked one folder up for instruction files (find .. -name AGENTS.md), outside the folder they were given; they wrote nothing there. Give it a copy of the folder, not the original.

Ask for the evidence, not just the answer

When the answer is a number, ask what could make it wrong and how it checked.

Plain: How much have we collected in fees this year?

With the practice: How much have we collected in fees this year? Before you answer, look through members.csv for anything that would make a simple sum wrong: duplicates, blanks, refunds, odd formats. List each one you find and say how you treated it. Then give the total, the number of members, and how you checked the total.

The right answer is $855.00; a plain sum of the column gives $900.00. Marked out of 5: the total, and naming the duplicate member, the blank fee and the refund.

Marks out of 5 Run 1 Run 2 Run 3
Plain 5 5 4
With the practice 5 5 5

The total was right in all six runs. The plain answers were three lines long and named the duplicate and the refund; one left out the member who had not paid. The careful answers listed every problem (two of the three in a table), gave the number of members (22, with the one who moved away explained), and checked the total two ways (“19 × $45 + 2 × $22.50 − $45 refund = $855.00”; “$900.00 − $45 duplicate = $855.00”). On this model the practice does not make the sum more right; it gives you the means to see that it is.

Say “only from these files”

Plain: What did the club pay for water in 2025?

With the practice: Answer only from the files in this folder. If the answer is not in them, say so, and say what the files do contain on the subject. What did the club pay for water in 2025?

The figure is not in the files. The minutes give 2026 only, and one gives $96.20 for electricity in 2025, a near miss. Marked right if it says the 2025 water figure is not there, wrong if it gives any amount.

Run 1 Run 2 Run 3
Plain right right right
With the practice right right right

All six runs, plain and careful, said the 2025 figure is not there. None made up a number or borrowed the electricity one. The careful runs also gave what the files do hold ($312.40 by September, $486.75 for the year, and “slightly below the previous year’s pace”); the plain runs gave one 2026 figure or none. OpenAI’s own wording for this boundary is worth copying: “Use only the supplied sources. Flag missing information instead of guessing.”

Correct it once, and have it write the rule down

When ChatGPT gets something wrong, tell it what was wrong and ask it to record the rule in AGENTS.md. The next session reads it.

The test, three times, in a folder with no instruction file: the notice task as above; then, in the same conversation, “That notice breaks our house rules … [the six rules] … Fix notice.md, then update your AGENTS.md so you don’t make that mistake again.”; then the notice task once more in a new conversation.

Run 1 Run 2 Run 3
First notice 8 7 8
After the correction 10 10 10
New conversation, reading the file it wrote 10 10 10

In all three runs it wrote an AGENTS.md with the six rules in its own words and added a rule of its own, worded differently each time, telling itself to check against the rules before saving: in one run, “Check every member communication against these rules before saving it”; in another, “Before saving a member communication, check every rule above and count all words, including the heading and signature”. The notices in the new conversation were the shortest of all the tests, 50 to 55 words. OpenAI gives the same advice: “When Codex makes the same mistake twice, ask it for a retrospective and update AGENTS.md.”

What did not hold up

Capitals and IMPORTANT. A common habit is to write rules as “IMPORTANT: YOU MUST…” and “NEVER, EVER…”. The same six house rules written that way scored 10, 10 and 10, exactly as the plain wording had. They made no difference because the plain wording already worked.

“Turn the reasoning up for better answers.” The plain fees question at high reasoning effort scored 5, 5 and 5, against 5, 5 and 4 at low: the same three-line answer, with the right total and neither the member count nor the check, because neither was asked for. It took about six seconds longer on average. Asking for the check in words worked; turning the effort up did not. OpenAI’s own advice is “Low for faster, well-scoped tasks”.

“Codex ignores CLAUDE.md; keep one AGENTS.md or link the two.” True, and it made no difference here. With the house rules in a CLAUDE.md and no AGENTS.md, Codex’s session records showed no instruction file loaded in any of three runs, as OpenAI’s page says (the records are summarised in the research folder). But in a folder this small it opened every file, read CLAUDE.md with the notes, and followed every rule. In a big folder, where it opens only what looks relevant, you cannot count on that. If you use both Claude and ChatGPT on the same folder, put the rules in AGENTS.md: Claude Code reads it when there is no CLAUDE.md in the folder (see the Claude guide).

“Codex doesn’t reliably follow AGENTS.md.” An OpenAI community thread says so. On these tasks the rules in the file held in nine runs out of nine (task 2 and two leads). OpenAI’s page documents two reasons a rule can go missing. Codex stops adding instruction files once all of them together reach a size limit (see The moving parts), so rules at the end of a long file can be lost. And a file called AGENTS.override.md is read instead of AGENTS.md in the same folder. Keep the file short; the same page’s example of checking what loaded is to ask Codex to “List the instruction sources you loaded.”

What the tests do not show

  • Chat, Work and the desktop app were not tested. Projects, memory, custom instructions and the plans’ limits come from OpenAI’s pages.
  • One model, at a light setting. GPT-6.1 Sol at low reasoning effort, the test account’s setting. Which plans get which model is in The moving parts.
  • The tasks were easy for this model. Plain and careful scored the same on three of five tasks; task 4 differed by one mark in one run, and task 2, the instruction file, by one to three marks a run. The practices showed up mostly in what ChatGPT told you about its work, not in whether the work was right.
  • No connectors, plugins or memory. They were switched off for every run, so these results say nothing about what ChatGPT does when it can reach your email or drive.
  • Three runs each. Enough to see that something happened in three runs out of three, not to say how often it fails.

Where your data goes

ChatGPT’s models run on OpenAI’s computers, so everything it reads is sent there to be processed, whether you upload it in Chat or let Codex open it on your own machine. Let It Open the Folder explains what that means for which folders to give it.

Training. OpenAI’s pricing page says, for Free, Go, Plus and Pro: “Content is used to train our models”, with “Opt-out available”. Business plans: “No training on your business data by default.” OpenAI’s help pages, which say where the switch is and what it covers, could not be read for this guide, so its name and place are not given here: look in the app’s settings.

Which rules apply. “When you sign in with ChatGPT, Codex usage follows your ChatGPT workspace permissions, role-based access control (RBAC), and ChatGPT Enterprise retention and residency settings.” With a developer API key, “usage follows your API organization’s retention and data-sharing settings instead”.

On your own computer. Codex keeps a transcript of each session in a hidden folder in your home directory (~/.codex), so you can resume it. You can turn that off in its settings file ([history] persistence = "none"). Its memories, if you turn them on, are files in the same folder. In the desktop app, “Syncing a conversation doesn’t upload the entire local folder or make it available to another computer.”

Connectors and plugins reach outside the folder. They are what lets ChatGPT read your email or drive, and OpenAI notes: “App and connector traffic is not controlled by the sandboxed-command network proxy or its domain allowlist.” A Codex set-up that has been used for a while can carry many. In these tests, a plugin bundled with the desktop app kept its tools loaded until plugins were switched off altogether (--disable plugins).

Settings worth changing:

  1. The training switch in the app’s settings, if you do not want your work used for training (its exact name and place were not checked here).
  2. Memory: read what it has saved (Settings > Personalization), and leave Codex memories off for private folders.
  3. Connectors and plugins: turn off the ones you do not need while working on private files.
  4. In Codex or the desktop app, keep the folder small: give it a copy of what it needs, not your whole Documents folder. Codex run from a script starts read-only; allow writing (--sandbox workspace-write) only in the folder you mean.

The moving parts

As read on OpenAI’s pages on 11 November 2026. Prices are as the Codex pricing page shows them, in US dollars; no Canadian price is shown, and tax is extra.

Free Go Plus Pro
Price a month US$0 US$8 US$20 US$100, US$200 or US$500
Codex models GPT-6 Luna, desktop app only GPT-6 Luna, desktop app only GPT-6.1 Sol and GPT-6 Luna As Plus, with more use
Codex on the command line Not listed Not listed Yes Yes
Chat Yes Yes Yes Yes
Work Limited, desktop app Limited, desktop app Desktop, web, mobile Desktop, web, mobile

Business is US$20 a user a month paid yearly, US$25 paid monthly, from two users.

  • Models: GPT-6 Astra (most capable), GPT-6.1 Sol (near Astra, cheaper), GPT-6.1 Terra (fast, for writing and images), GPT-6 Luna (most efficient). GPT-5.5 left Codex for ChatGPT accounts on 14 October 2026.
  • Limits: Work and Codex share one allowance. OpenAI’s estimate for Plus is 15 to 160 local messages every five hours on GPT-6.1 Sol, “not fixed message limits”; weekly limits may also apply. Pro has no five-hour limit. See yours at chatgpt.com/codex/settings/usage, or /status in Codex.
  • Codex CLI: these tests used 0.166.0. Version 0.167.2 came out on 11 November 2026, after the tests.
  • Instruction files: Codex stops adding them once all of them together reach 32 KiB, a default that can be changed.
  • Reasoning effort: the tests ran at low, the test account’s setting. A Free or Go account gets GPT-6 Luna in Codex, and only in the desktop app.
  • What the tests used: 51 Codex turns for the tasks and leads, between 13 and 34 seconds each, usually about 77,000 input tokens a turn, most of it the same cached set-up. On a plan this comes out of the shared allowance, not a bill.

What changed

  • 12 November 2026. Made consistent with the Claude and Grok guides: the practices in one order, the task 5 table, one paragraph on how the tests were run, what the runs touched outside the request, and /plan marked untested; the size limit and the paid-plan model moved to The moving parts.
  • 11 November 2026. Corrected after review: task 3’s plain runs re-marked with the plain-prompt check (all passed); the reasoning effort described as the test account’s setting; quotes and counts matched to the test log.
  • 11 November 2026. First published.

Sources

OpenAI’s pages, read on 11 November 2026. The research notes, the test log with every mark, and each run’s output are in the accompanying folder. The made-up folder, prompts and answer key shared by all the assistant guides are in the common folder.