The model is being retired. Your order inbox is staying open.
Matías Bonvin· Updated
Imagine an email arriving from your model provider: the version behind your order-processing assistant will be withdrawn on a stated date. A replacement is available.
The technical change looks small. Someone updates a setting, sends a sample request and gets a sensible response. Meanwhile, customers keep sending attachments with old product codes and delivery instructions in the wrong field.
Those are the cases the replacement needs to handle. A successful connection does not establish whether the workflow can rely on the answers.
Find every process using that version
Start with the exact model identifier in the retirement notice. Find where it is used, including scheduled jobs that nobody opens in a browser. Record the cutoff date from the provider's current documentation and assign someone to own the migration.
For each affected workflow, note the business owner, the people relying on the output and the action that follows it. An assistant drafting an internal summary can fail differently from one preparing an order for release.
Check who receives future notices. A subscription email going to a former contractor can turn a planned retirement into a surprise outage.
A row per workflow can hold the model version, account owner, retirement date, test location and fallback. Assign someone to keep it current.
Give the replacement your difficult cases
Use authorised historical examples in a safe environment. Remove sensitive data where necessary, while preserving the details that make the cases difficult.
In our order-processing example, the comparison should include an unfamiliar product reference and an attachment that contradicts the email. Include an incomplete order, too. The expected behaviour may be to ask a person rather than guess.
Define acceptance before seeing the new model's output. Which fields must be extracted correctly? Which uncertainty must stop the process? Which actions remain subject to approval?
Run the old and proposed versions on the same cases where the old version is still available. Inspect the result that reaches the business system, not only the text returned by the model. A correct answer in a different format can still break the next step.
Repeat cases where variation would matter. Record errors, review effort, response time and cost under representative conditions. An average score can hide a rare mistake that your operation cannot accept, so list those failures separately.
Decide how work continues if the test fails
The replacement may need more work than the remaining time allows. Decide in advance what happens to incoming requests in that situation.
For the illustrative order workflow, you might route new orders to an existing manual queue while keeping the assistant out of the release path. Name the person watching the queue, the capacity available and the point at which customers need an updated lead time. Test that route before you need it.
Switching back to the old model can be an option before retirement. After the provider removes it, a rollback that depends on that model will fail. Your fallback must remain available beyond the cutoff.
Whether you choose temporary manual handling or a tested alternative, name someone authorised to activate it. Keep a record of processed cases so restarting does not create duplicate orders.
Move a bounded part of the work first
Once the replacement meets the agreed criteria, start with a controlled slice and inspect the outcomes before widening its scope. Keep the original business approval boundaries intact. A model change does not give the assistant new authority.
The work also needs a budget and an owner after launch. Ask who tracks notices, maintains the test cases and pays for a required migration. Agree those responsibilities when buying the workflow, while there is still time to price and staff them.
At Solve IT, the implementation scope can include those operating dependencies alongside the first release. A Strategy Session can start by tracing one workflow from the incoming request to the model it depends on, then identifying what would keep it running during a change.
Bring the name of the model your order inbox will need on Monday.