Sequential decision-making problems and acting in an environment, where the right action depends on the one before it and the current environmental state.
Some decisions, fast or slow, aren't one-shot predictions, but form part of a sequence, where today's action changes the state you'll decide from tomorrow. That can be framed as a reinforcement learning problem, instead of a supervised one which may have pre-determined labels on good/bad behaviour.
The challenge here becomes reward design to specify what "good" means precisely enough to avoid reward hacking and loosely enough to not restrain agents unnecessarily; and building a simulation environment realistic enough that a policy trained in it transfers, or generalises, to the real system.
Where live exploration carries real cost or risk, we favour offline RL and off-policy evaluation: learning from historical decision data and rigorously estimating a new policy's performance before it's ever deployed. Avoiding learning by trial and error in production by doing it in the training environment/data.
Deployment is staged and monitored — a policy earns expanded autonomy as its real-world performance validates the offline estimates.
How we approach it
Defining state/environment space, action space, and a reward function, or functions, that properly encodes the business objective/agent task, stress-testing for scenarios in which a policy could exploit rewards to score well without undertaking the desired actions.
Building or calibrating a simulation environment realistic enough for policies trained using it to adhere to desired behaviours in the real system — or, where a simulator isn't feasible, working directly from historical data and decision logs to inform agent/policy decisions.
Selecting an algorithm matched to the action space (discrete or continuous), data availability and volatility, and whether online exploration is safe and suitable — online RL where it is, offline RL from logged data or simulated environments where it isn't.
Estimating how a new policy would perform against the real objective before it is involved in production, using the historical data for backtesting, with confidence intervals where appropriate.
Rolling out with guardrails and a fallback policy, expanding the trained agent/policy's autonomy only when live results confirm offline estimates — and monitoring for reward hacking after launch, not just before it.
What the RL training loop looks like
Agent, action, environment, reward
The agent acts, the environment responds — the policy improves from the reward signal, one cycle at a time.
Tell us about the problem and we'll scope it together — no obligation.
Start a conversation →