Businesses are becoming more ambitious about where they use AI.
The conversation is moving beyond generating content and answering questions. Increasingly, organisations are looking at systems that can help carry out work: investigating financial disputes, supporting laboratory processes and handling steps in complex workflows.
That creates a different challenge for technology leaders.
When a tool gives you an answer, you can check it. When an AI agent completes a sequence of actions, the business also needs to understand how it reached the result, where it may have gone wrong and how its performance can be improved.
In other words:
How do you know the agent is doing a good job?
This week's New York technology developments offer a useful view of that challenge.
Crux Labs: evaluating the work of AI agents
On 29 October, New York-based Crux Labs announced that it had emerged from stealth with more than $6 million in funding.
The company says it is building a platform to help organisations evaluate and improve AI agents using human judgement.
Its proposition reflects a question many technology leaders will recognise: once an agent is operating inside a real workflow, how do you decide what "good" looks like?
A task can appear complete while still producing an outcome that is inaccurate, inconsistent or unsuitable for the customer.
In some cases, the problem will be technical. In others, the business may never have clearly agreed what a good decision looks like in the first place.
That makes evaluation a leadership issue as well as an engineering one.
The CTO may own the technology, but the standards may need input from the people closest to the work: customer operations, finance, legal, risk and other subject-matter experts.
The questions include:
- What does a successful outcome look like?
- Which mistakes matter most?
- When should an agent's work be reviewed?
- How will the organisation learn from errors?
- Who has the authority to change or stop the process?
The leadership question: Who defines success for an AI agent, and how does the business identify and learn from decisions that fall short?
Source: Crux Labs funding announcement, 29 October 2026.
FinTech: AI agents move into financial disputes
On 28 October, financial services AI company Casap announced a $56 million Series B, led by Base10 Partners, with participation from Experian and Synchrony.
Casap builds AI tools for financial services workflows, including disputes and fraud-related processes. Tech:NYC lists Casap as a New York business.
Financial disputes are an example of why the next stage of AI adoption involves more than speed.
A business may want to reduce the time its teams spend reviewing information. It must also consider whether a decision is accurate, how it can be explained and what happens when a customer or investigator challenges it.
For a technology leader, that means working across product, operations and risk. The questions are practical:
- Which parts of the process can be handled consistently by an agent?
- Which cases should be passed to a human?
- What information needs to be recorded?
- How will the team spot recurring errors?
- How will the business measure whether the process is working?
For a founder or investor, a funding round like this is a market signal: businesses are investing in AI products designed to perform work inside financial processes.
It does not, on its own, tell us whether a company is hiring a CTO. It does show the type of product and operational challenge senior technology leaders in FinTech may increasingly be asked to address.
The leadership question: Can the technology team improve speed and consistency while giving the business confidence in the decisions being made?
Sources: Casap funding announcement, 28 October 2026; Tech:NYC digest, 2 November 2026.
HealthTech and life sciences: connecting AI with laboratory automation
On 29 October, Long Island City-based Opentrons announced a $40 million Series E, led by Avego, with additional investment from BroadOak Capital Partners.
Opentrons develops laboratory automation technology and says the investment will support the development of its robotics, software and data infrastructure.
The significance here is the combination of software, data and physical processes.
In a laboratory, an error is not simply an incorrect answer on a screen. The way an experiment is designed, executed and recorded can affect the usefulness of its results.
That creates a demanding environment for technology leadership. Engineering teams need to work closely with scientists and laboratory users to understand where automation helps, what needs to be documented and how systems fit into existing work.
It also raises a broader point for founders building in specialised markets.
Technical capability matters. So does the ability to bring together people who understand different parts of the problem.
A technology leader may be responsible for engineering delivery, but success can depend just as much on how effectively they listen to the people using the product and translate their needs into decisions.
The leadership question: Can the engineering team explain how its technology supports the people doing the work—and how they will know when it has performed as intended?
Source: Opentrons funding announcement, 29 October 2026.
⛏️ From the Coal Face: AI needs an engineering mindset
In conversations with technology leaders this week, I was struck by two different perspectives on AI.
One engineering leader described wanting to work in an organisation that embraces AI while retaining an engineering-first mindset. His concern was the temptation to drop everything and rush towards agents without enough thought about how the technology fits the work.
Another leader described using AI-assisted coding to help build a tool around a commercial problem. What stood out was the combination of business understanding and technical delivery: knowing what the user needed was as important as being able to build it.
These are individual experiences, not a survey of the market. But together they raise a question for founders and boards:
Are we asking our technology leaders to deploy AI—or giving them the space to decide where it will create value and how the work should be assessed?
Moving quickly is valuable. So is being able to explain what has been built, whether it works as intended and who is responsible for improving it.
The strongest technology leaders need to be comfortable with both: exploring what AI makes possible and applying the judgement to use it well.
What this means for your next technology appointment
The developments above do not all call for the same type of technology leader.
A FinTech scaling a product for financial institutions may need a CTO who can balance product delivery, security, customer requirements and risk.
A laboratory automation business may need someone comfortable bringing together software, hardware and scientific teams.
An AI company building products for other businesses may need a leader who can turn an emerging capability into a reliable product that customers can understand and trust.
In each case, the starting point should be the business challenge.
Before approaching the market, I would want a founder or board to consider:
- What needs to be working in 12 months that is not working well enough today?
- Which decisions should the technology leader own?
- Where will they need to work alongside operational or domain experts?
- How will the business measure quality, commercial value and risk?
- What team and authority will the leader need to deliver the outcome?
Those answers will tell you more about the person you need than a long list of programming languages or AI tools.
The takeaway
The next phase of AI adoption will put more emphasis on how work is evaluated, challenged and improved.
For technology leaders, that means bringing technical teams and business experts together around a shared understanding of what success looks like.
For the businesses hiring those leaders, it means defining the outcome before defining the job title.
Because the question is no longer simply:
"Can we use AI to do this?"
It is also:
"Should we, how will we know it is working, and who is accountable for the result?"
If you’re responsible for a pivotal hire, let’s talk.
Fifteen confidential minutes on the role, the market and whether we’re the right firm for it.