Patient safety depends on making critical care decisions with complete clinical context, yet too many patients still reach intersections of major care journeys, such as surgery or cancer care, with key clinical context still missing. Undocumented medications, overlooked devices, outside diagnoses, and fragmented specialist records create preventable risk at the moment care teams are expected to act with confidence. Complete patient readiness addresses that gap by ensuring the right information is available early enough to guide decisions, leading to improved quality of care and clinician satisfaction in delivering that care.

Think of a patient that arrives for a procedure and late in the process the team discovers they are taking a blood thinner that was never documented correctly, or that they have a pacemaker that wasn’t identified or tracked when planning the case. Suddenly the case becomes a scramble. The care team is forced to choose between delaying the case, or proceeding under less-than-ideal conditions, neither of which is a good setup for high quality care.

In another setting, a patient begins cancer treatment with a significant cardiac history that was scattered across outside records and never fully surfaced before therapy began. The treatment plan moves forward with a cardiotoxic chemotherapy, and that important piece of missing historical information directly drives harm to the patient when it is discovered too late or when the patient has a complication with treatment.

Unfortunately, patient readiness has been easier to describe than to deliver. Clinicians and patients often have an ideal patient journey in their heads, and become frustrated when that ideal isn’t achieved. Medical records are scattered and hidden across multiple EMRs and not accessible to care teams or patients. Healthcare reimbursement has incentivized “good enough” at high volume, instead of rewarding high quality personalized care. And meanwhile, patients continue to get sicker and more complicated.

However, these three trends are now converging as tailwinds to make patient readiness a priority for the healthcare system to deliver more efficient, higher quality care.

1. The cost of generating accurate patient context is collapsing

The 21st Century Cures Act established TEFCA, a nationwide exchange framework now connecting more than 400 hospitals, 5,000 physician offices, and over 100,000 individual clinicians through a common agreement, while FHIR standardized the format for how that data moves between systems. The burgeoning focus on information blocking now makes it illegal for institutions to sit on data that should be accessible to the care team. None of this is perfect, and anyone who has tried to use an HIE tool as a side tab in an EMR to review outside records knows exactly how much work remains. But the foundation is now there in a way it simply was not five years ago.

This is where AI can enter the picture, with the ability to turn accessible data into usable clinical context at a speed and cost that no manual process can match. What previously required a trained and experienced clinician spending most of an afternoon assembling a preoperative picture from fragments, can now be generated in a fraction of that time, and often with better completeness. At Orchestra, we pull clinically relevant summaries from outside cardiology notes, specialist documentation, medication histories, and imaging reports across disconnected systems in seconds. The economic barrier that kept serious patient readiness work out of reach for most facilities is coming down, and it will keep coming down.

2. Payment models are beginning to reward knowing your patient before delivering care

Fee for service models in healthcare have had one major incentive structure, and meticulous patient readiness is not part of it. Fee for service rewards seeing as many patients as possible, regardless of the clinical status of the patient. Because of this, there is little to no reimbursement or revenue generated for triaging patients or covering the cost of thoroughly navigating records. This cost often ends up falling on a different part of the system than the one that generated the revenue from seeing the patient or booking the case.

That is continuing to change. The TEAM model launched at the start of this year, creating mandatory shared accountability between hospitals and surgeons for surgical episode outcomes across joint replacements, spine surgery and major bowel procedures. Meanwhile, procedures continue to come off the inpatient only list, moving into ASC and outpatient settings where margins are thinner and there is no hospital DRG to absorb the cost of a preventable complication or late cancellation. Finally, the recent ACCESS model is extending that logic further, by creating a canvas for outcome aligned payments of chronic conditions as opposed to surgery or acute conditions.

This shift will only continue, and patient readiness will be the key that unlocks margin in this environment. Capitated risk that isn’t identified can’t be managed.

3. The patients coming through the door are getting consistently more complicated

A 2024 study tracking 3.4 million hospitalizations over 15 years found that patients were 50% more likely to have multimorbidity, 82% more likely to be on polypharmacy, and twice as likely to require treatment for five or more acute medical issues compared to the beginning of the study period. A parallel analysis of 107 million hospital admissions in England found a 35% absolute increase in acute admissions among patients over 65 managing three or more chronic conditions, representing nearly a million additional complex hospital episodes annually compared to 15 years prior. In the United States, the prevalence of adults with two or more chronic conditions rose from 45.7% in 1988 to nearly 60% by 2014. Among adults over 65, multimorbidity prevalence is now approaching 92%.

Population aging explains some of this, but only some. Even after controlling for age and sex, the burden of extreme multimorbidity continues to climb. More patients, more medications, more specialists involved across more institutions, and more opportunities for something critical to fall through the cracks between those specialists before the patient lands in your preop area or clinic.

As the patient population gets more complex every year, the amount of time and reimbursement for the same visits and procedures is staying the same if not decreasing. The expectation that good clinical judgment in the moment will catch everything that matters is increasingly unrealistic. We can’t afford to keep doing things the same way in the face of this increasing medical complexity crunch.

Patient readiness is no longer optional

For health systems, patient readiness is a way to improve safety, reduce friction, and better match resources to patient need. For payers, it is a way to identify risk earlier, avoid preventable downstream utilization, and support more appropriate care navigation. For clinicians, it is a way to spend less time assembling the chart by tasks that are well below their level of license and spend more time making decisions that require their training.

Healthcare has talked for a long time about getting the right patient to the right care, in the right place, at the right time. Patient readiness is that operational layer that will make it possible.

Medication reconciliation should be easy to automate. Every prescription is tracked. Every refill is logged. Every dose is documented. And yet, across healthcare today, “med rec” remains one of the most error-prone, time-consuming tasks clinicians perform.

The disconnect reveals something important about healthcare, but really systems as a whole: the hardest problems aren’t computational—they’re human.

A med rec aims to do exactly what its name suggests: reconcile the provider’s chart with the medications a patient is actually taking. Straightforward in theory. In practice, this is where the neat digital trails break down—and where AI can genuinely help, if applied correctly.

How Prescription Medication Is Supposed to Work

The standard process for prescription medication looks clean on paper:

1. Patient encounter: A patient meets with a healthcare provider to discuss symptoms and health concerns.

2. Prescription: The provider prescribes medication as part of a treatment plan.

3. Pharmacy processing: A pharmacy receives the prescription, doses the medications, and packages them.

4. Dispensing: The patient picks up the medication from their preferred pharmacy.

At every step, the medication, quantity, and dose are tracked and reconfirmed. Every refill is logged. Prescriptions expire, so they can’t be used indefinitely. With this much documentation, reconciling these records should be trivial, right?

Spoiler alert: It’s not.

Where things break down

The problem is that patients don’t fit into neat boxes—and they aren’t healthcare professionals.

Medications get picked up but are not taken. Adherence is notoriously poor. Patients stop antibiotics when they feel better, not when they’ve completed the course. They skip doses to stretch expensive medications. They abandon treatments with side effects nobody warned them about.

Medications aren’t taken as prescribed. Some patients halve doses to save money. Others double up when they feel worse. PRN (”as-needed”) medications add another layer of ambiguity—how often, exactly, is “as needed”?

Patients don’t know what they’re taking. This is especially common among elderly patients managing multiple conditions, but it occurs across all demographics. The confusion between medication names, active ingredients, and brand names is pervasive.

The Lisinopril Problem

Consider a patient taking lisinopril for hypertension. They might tell one provider they take “Prinivil” (a brand name), another that they’re on “that blood pressure pill,” or “an ACE inhibitor”, or some collection of our personal favorites, “the green one”, “the round ones”, or “the ones that look like pills”. If they switched pharmacies and received a different generic manufacturer, the pill might look completely different—new shape, new color—leading them to believe it’s a new medication entirely.

Now multiply this confusion across a patient taking eight medications. Some of them stopped months ago. Some take it differently than prescribed. Some of them received as samples and were never filled through a pharmacy at all.

This is why a clinician can spend twenty minutes, if not more, on a med rec for a single patient—and still get it wrong.

The Data We Have (And Its Limits)

There’s more data available today than ever, but each source has significant gaps.

Health Information Exchanges (HIEs) aggregate prescription data and past reconciliation records across health systems. But HIEs are only as good as their network coverage, and they capture what has been prescribed, but not what was taken. We focus on what a patient is actually taking.

Pharmacy networks like Surescripts provide real-time access to dispensing data—fill dates, quantities. But again, they capture what was dispensed, not what was taken, and miss medications from non-participating pharmacies, mail-order services, or samples.

Patient interviews remain the gold standard for understanding actual use. But patients forget, misunderstand, and misreport—and interviews are time-intensive for already-stretched clinical staff.

None of these sources alone provides ground truth. Effective med rec requires triangulating across all of them.

How LLMs Can Actually Help

While clinicians are still necessary to determine reality in the context of a med rec, we have found several strategies that leverage both LLMs and other tools to reduce the time they need to spend getting there dramatically.

Pre-population and record scanning. AI can aggregate data from HIEs, pharmacy networks, and historical records to pre-populate a medication list before the patient encounter begins. Natural language processors can extract medication mentions from unstructured clinical notes, and other free text. The goal isn’t perfection—it’s giving clinicians a 70-80% complete starting point instead of a blank slate.

Risk stratification. Not all medication discrepancies carry equal risk, therefore should not be treated equally. Using AI to detect medications doesn’t need to be 100% accurate today, although one can dream, because flagging for a multi-staged triage approach will still dramatically improve results in many cases, such as:

• Patients on anticoagulants where dosing errors cause bleeding or stroke

• Patients with recent hospitalizations where medications frequently change,

• Detecting discrepancies between prescribed and filled medications suggesting non-adherence.

By bringing a lot of this to the front without the clinician needing to spend time digging. Clinicians can focus their time on the patients and medications that matter most.

Pre-visit verification. Emerging AI voice agents can conduct structured phone calls to verify medication lists with patients before appointments—reading back current medications, probing for over-the-counter supplements and discontinued drugs, and documenting responses in structured formats for clinical review. This shifts routine verification work to AI while preserving clinician time for the complex cases.

The Payoff

When implemented thoughtfully, AI-assisted med rec delivers concrete improvements—less time on rote data gathering, freeing clinicians to focus on actual clinical judgment. High-risk items are caught first rather than discovered by accident—fewer medication errors, which remain a leading cause of preventable harm.

Conclusion

Med recs resists automation not because the data doesn’t exist, but because patient behavior is inherently messy. Until we’re able to credential an autonomous system completely, the next best step is to augment the current workflow with intelligent data aggregation, risk stratification, and preliminary verification, removing low-level tasks from it.

Let AI handle the tedious triangulation so clinicians can focus on the judgment calls that actually require their expertise. That’s not a revolution—it’s just good engineering applied to a stubborn problem.

What is a Review of Systems?

At its core, a Review of Systems (“ROS”) is a top-to-bottom assessment of a patient’s physical status. it is, by design, a systematic checklist (doctors love a checklist), with the primary goal of providing a consistent and thorough starting point from which the provider can begin treatment: starting large (which body systems are affected?) and narrowing to specific issues and concerns the patient may be facing.

The logic flows in two directions:

• Large to small: Reviews begin with location-specific systems—cardiovascular, respiratory, gastrointestinal—before drilling down into discrete symptoms within each.

• Broad to specific: The review targets general conditions first, then narrows based on patient responses. A “yes” to fatigue opens different pathways than a “yes” to chest pain.

This structure exists for good reason: it provides a standardized starting point that any clinician—NP, PA, or MD—can work from.

Why It Matters

The ROS serves as the foundation for the entire patient journey; it influences diagnoses, treatments considered, and ongoing care that may be necessary. If it’s done poorly, everything else that happens afterwards is affected.

A data scientist would call this garbage-in, garbage-out. The same principle applies in the clinic: a rushed or incomplete ROS doesn’t just slow things down, it undermines ultimate success.

The Problem with the Status Quo

Traditionally, ROS has been rote, generic, and one-size-fits-all for most patients and encounters. The same questions, with mild variations, regardless of what we already know about a patient, which is usually minimal anyway.

Updating a Classic

We’re now able to do something different.

Starting with the Data

It begins with the data. For patients presenting for complex treatment, we can now prefill more than half of a ROS, usually the boring bits, before the encounter.

When does this work:

To find data, data must exist; there’s no crystal ball for a patient’s health. A healthy patient who doesn’t require much treatment will likely only benefit mildly from this; however, patients undergoing complex treatment have typically engaged extensively with the healthcare system beforehand. You don’t jump straight to complex work for many reasons, risk and cost being the primary two. As patients move through the levels of treatment triage, more and more is learned. The amount of data, particularly recent data, we have on a patient grows proportionally, so much so that we have an inside joke at the office that almost anyone on our team, regardless of training, can determine someone’s ASA score simply by looking at how much recent data we have on a patient.

When you’re in the American healthcare system, you don’t expect things to move quickly... and then when it starts to get a little bit efficient, it’s actually quite disconcerting. — Hank Green

In the past, those records were used sporadically, if at all. Now, we can fully capitalize on the work of previous providers, and the benefits compound:

• Faster treatment: Less time spent re-asking questions the patient has already answered

• Better care: Clinicians start from a more complete baseline, with more time for the questions that actually matter

Data is only the beginning

Having access to comprehensive patient data is necessary, but far from sufficient. The real work begins when data is combined, cross-referenced, and thoughtfully processed against preferenced-guidelines. Patient histories scattered across different providers, labs, imaging centers, and specialists need to be reconciled to remove conflicting information and standardized for efficient use.

How we accomplish this is a topic we’ll address in depth later. For now, know that the Orchestra Clinical Data Network gives us the raw ingredients to build new journeys.

Responsible use of LLMs within care journeys

A cornerstone of efficiently augmenting ROS today lies in fundamentally understanding how best to use LLMs to both structure and detect relevant information. That conerstoner includes a core understanding that LLMs are not intelligent and are fundamentally an autocomplete tool, albeit an incredibly powerful one.

Where we’ve seen success

Detection: As fuzzy searchers, LLMs are the pinnacle of modern implementation. They can surface relevant information buried in thousands of pages of clinical documentation. They handle misspellings, abbreviations, and the idiosyncratic shorthand that litter clinical documentation.

Structuring: We’ve built systems that ingest large volumes of unstructured data and, with human guidance, rapidly transform it into structured, actionable information. Free-text notes become coded diagnoses. Scattered observations become timelines. This structured output then feeds deterministic systems that can apply consistent clinical logic at scale.

Where we’ve seen inconsistencies

Prescriptive Language: LLMs love telling us what to do, and if you let it, it will happily and confidently develop and regurgitate a complete care plan for any individual patient chart you point it at, with or without guidance. EMRs are, in a very real way, legal records, which makes prescriptive outputs from LLMs at best risky, and at worst, fraud.

Not all regulations are bad

We said that there is no crystal ball for a patient’s health, which might lead one to wonder why this data hadn’t been used previously. Honestly, it simply wasn’t accessible.

Until recently, leveraging data outside of your system (and sometimes even within the system) Much access to this data was typically walled off within private EMRs, under the guise of privacy and protection, which complicated sharing of data even within a single system, however, in 2021 we started to see the beginnings of change, through the implementation of the Cures Act which was signed into law in 2016 and went into effect in 2022.

Brief Summary The 21st Century Cures Act mandated the creation of TEFCA (Trusted Exchange Framework and Common Agreement), a nationwide framework for secure health information exchange. TEFCA establishes a standard set of technical and legal requirements to enable different health information networks to connect and share data seamlessly across the country.

Who It Impacts

• Healthcare providers and hospital systems

• Health information exchanges (HIEs)

• Health IT vendors and EHR developers

• Patients accessing care across different networks

• Qualified Health Information Networks (QHINs) participating in the framework

Benefits

• Enables nationwide health data exchange across previously disconnected networks

• Reduces burden on providers by standardizing exchange requirements

• Improves patient care continuity when receiving treatment from multiple organizations

• Streamlines access to patient records across state lines

• Reduces duplicative testing and administrative costs

• Creates a single on-ramp for organizations to connect to various networks

Better foundations lead to better results

The ROS is a critical cornerstone of effective medical practice, but its potential and efficacy have always been limited by the tools available. Coupling clean, structured data with thoughtfully deployed systems enables more effective treatment with far less effort or time.

Building a Patient Readiness Platform

Every healthcare organization performing complex procedures needs to understand a patient’s current clinical condition, both on its own and in the context of the specific procedure a provider wants to perform.

This requirement is universal across healthcare delivery. The implementation of it varies significantly.

What is Patient Readiness?

Patient readiness means determining whether a patient’s current condition is sufficient to safely proceed with a specific procedure. It’s deliberately broad because it has to cover a lot of ground:

• Physical status

• Medications

• Procedural requirements

• Logistical requirements

• Physician preferences

Physical Status

This includes the concrete medical data: lab results, vital signs, imaging scans, current diagnoses, and recent health events.

For surgical patients, you’re also assessing cardiac risk, lung function, nutritional status, and blood sugar control. All the things that determine whether someone can handle the physical stress of a procedure.

Medications

Medication management is trickier than it sounds. Medications need to be checked against the current guidelines and cross-referenced between both facility and physician preferences. Some drugs need to be held, some continued. Some require additional testing first to see if it’s safe.

And here’s what makes this complicated: medical guidelines change as research advances. Standard guidelines five years ago might be outdated today.

Procedural Requirements

These are required tests, approvals, and prep work that need be completed before the procedure can take place, as the name suggests, This list isn’t universal and depends heavily on the procedure itself, the patient’s risk factors, and how those two play together, how the facility operates, and the preferences of the surgeon and anesthesiologist running the show.

Logistics

Logistical requirements cover issues such as patient availability and recovery planning. You’d be surprised how often patients get cancelled because they don’t have a ride home.

Physician Preference

The final factor is provider preference. Physician groups are called practices, and we’d argue the term is not coincidental. Like athletes and artists, medical professionals have their own styles and preferences. These shape the care plan and influence everything from preparation through execution.

The Output is Simple. The Process is Not.

The frustrating part? After all this complexity, patient readiness boils down to three options: go, delay, or cancel.

Knowledge Isolation

Critical process knowledge often lives with a small number of experienced staff. When these people leave or call in sick, process reliability drops.

Integration Gaps

The information you need for readiness decisions exists across multiple systems. Someone has to manually aggregate, reconcile, and interpret all that data.

Orchestra: Automating Patient Readiness

Orchestra is a platform that automates patient readiness evaluation. It integrates with existing clinical workflows and applies organization-specific rules to determine who’s ready and who’s not.

Data Aggregation

Orchestra connects to your existing clinical systems and pulls together everything needed for readiness evaluation. We combine labs, prescriptions, orders, scheduled appointments, documentation, as well as structure unstructured sources like doctor’s notes, patient preference, and imaging together.

Rule Application

The system applies your organization’s clinical rules and risk thresholds to patient data. These are your guidelines, not someone else’s standards imposed from outside.

Status Determination

For each patient, Orchestra produces a readiness assessment that determines which criteria the patient meets. Which are still outstanding, and where the patient has breached the threshold.

Exception Surfacing

When issues need human attention, the system routes them to the right staff with relevant context. No more manually checking on every patient.

The Complexity Spectrum

Not all patient readiness evaluations are created equal. They range from simple to highly complex.

Complexity depends on procedure risk, the number of clinical domains involved, and how much coordination is required.

Let’s look at both ends of the spectrum.

Low Complexity: Blood Donation

Blood donation sits at the simpler end, when you donate blood a donation centers screen for defined exclusion criteria before collection:

• Iron levels, to check for anemia

• Recent travel

• Current medications

• Vital signs

• History of specific medical conditions

All of this can generally be done on site or remotely with high accuracy and low cost. Because the procedure is complex and the guidelines are straightforward, we’ve been able to scale donating blood nationally as somethign that can occur easily without an appointment during a lunch break.

High Complexity: Orthopedic surgery

At the other end of the spectrum we have something like orthopedic surgery, which is generally where we spend most of our time.

Preparing someone for total joint replacement requires extensive preparation across several disciplines.

Cardiac/Pulmonary Assessment

Complex surgeries place enormous strain on the body, which stresses most systems including your heart and lungs. To understand your body’s ability to withstand this stress tests like EKGs and chest X-rays are often to used to develop an understanding of those systems.

Pulmonary Evaluation

This might include lung function testing and chest imaging. Patients with chronic issues need to be identified so they can optimized prior to surgery.

Medication Management

This requires coordination across multiple drug classes:

• Anticoagulants need specific hold periods based on how long the drug stays in your system and the patient’s clotting risk

• Antiplatelet agents are managed based on bleeding risk and why the patient takes them

• Blood pressure medications are continued or stopped based on anesthesia protocols

• Diabetes medications get adjusted for fasting

• Herbal supplements and over-the-counter medications are reviewed for surgical risks

Preoperative Testing

Testing follows a schedule tied to the planned surgery date. Lab values obtained too early must be repeated. Testing done too late might delay surgery.

Clearances

Specific system assessments provide more detailed recommendations on how to proceed with the procedure that can be drawn upon for a final decision. Each of these are respective, and sometime pejoratively, know as clearances. These might be required from cardiology, pulmonology, endocrinology, or other specialties, depending on the patient’s other health conditions. Primary care doctors often document that the patient is medically optimized.

Who’s Involved in Complex Readiness?

Preparing a patient for major surgery involves a lot of people:

Surgeons define procedure-specific requirements and make final calls about patient suitability.

Surgical coordinators and preoperative nurses track completion status, gather missing information, and communicate with patients.

Anesthesiologists evaluate airway anatomy, cardiac and pulmonary function, and medication considerations.

Primary care physicians provide medical history, manage chronic conditions, and document optimization.

Medical specialists address specific risk factors in their domain—cardiologists for heart risk, lung doctors for breathing optimization, and so on.

Pharmacists review medications and identify potential interactions or concerns.

Laboratory and imaging staff complete time-sensitive testing.

Schedulers coordinate operating room time, bed availability, and staffing.

Case managers address post-discharge needs and discharge planning.

Patients and caregivers complete assigned prep tasks, provide updated information, and confirm they understand instructions.

Each person operates within their own system and workflow. Information exists across multiple platforms.

The surgical coordinator typically functions as the central hub, manually tracking what’s done and what’s not, following up on outstanding items, and maintaining awareness of overall readiness.

This model is labor-intensive. It depends heavily on individual coordinator capacity and institutional knowledge.

It doesn’t scale well with patient volume.

What Happens When the Process Fails

Patient readiness failures have real consequences.

Same-Day Cancellations

These happen when readiness issues are identified too late. The operating room slot is lost or underutilized. The patient must be rescheduled, repeating parts of the prep process.

Studies estimate same-day cancellation rates of 5-10% at many institutions. Inadequate preoperative preparation is a leading cause.

Procedural Delays

These occur when outstanding items are found on surgery day, but can be addressed quickly. The case starts late, affecting all subsequent cases on the schedule—staff overtime increases. Patient and family experience suffers.

Suboptimal Patient Condition

This happens when fixable issues aren’t identified or addressed before surgery. Patients with uncontrolled diabetes, unoptimized anemia, or poorly managed heart conditions face higher complication rates.

Adverse Events

In severe cases, missed contraindications can cause preventable harm. Medication interactions, unrecognized allergies, or unidentified health conditions can result in serious problems.

Staff Burden

As coordinators manage increasing patient volumes with manual processes, the burden accumulates. Turnover in these roles creates knowledge gaps and training requirements.

These consequences are largely preventable. You just need to identify readiness gaps early enough to fix them.

Application

Orchestra addresses the operational challenge of patient readiness by providing automated, configurable evaluation against your organization’s criteria.

The platform doesn’t replace clinical judgment. It won’t make decisions for you.

What it does do is provide clinicians and coordinators with consolidated information and systematic status assessment. This lets them focus their attention on items that actually require human decision-making.

Patient readiness is a prerequisite for procedural care. It’s the first question you need to answer.

Orchestra provides the infrastructure to answer it consistently, every time.

“If we’d had Orchestra last year when we did this case, he’d probably still be alive.”

This was a comment made by a customer after we completed their first live demo.

By live demo, I mean we took in real data, with real patients and real information available then, and showed them what might have gone differently if they’d used Orchestra. As a sales tactic, we’ve found it to be effective.

This time, however, was slightly different, as we were conducting the demo as a postmortem, literally, for the first time. The patient in this scenario had died intraoperatively after experiencing cardiac arrest. The question was: why?

Preventing scenarios like this is a core reason Orchestra Health exists. As a patient readiness platform, we are responsible for understanding what is happening, who it’s happening with, how it will happen, and — most importantly — where things may go wrong, when we’re working optimally.

In this case, the patient had suffered a cardiac incident under anesthesia five years prior. The incident had been reported, documented, and shared … only to be subsequently buried by five years of patient data, clinic notes, lab results, imaging studies, referrals, and medication changes. The signal was there, it just wasn’t surfaced.

How Do You Test an Agentic, non-Deterministic System When Life Hangs in the Balance?

Orchestra allows health systems to leverage agentic systems to understand what needs to happen next in a fraction of the time it takes with conventional means. For some of our customers, it is integral to their entire operation.

This creates a problem: these systems are non-deterministic. The same input does not always produce the same output. Model behavior changes with updates from our sub-processors. Context windows, token sampling, and inference variability all introduce unpredictability.

We need to iterate on these systems while preventing regressions pre-deployment and identifying failures post-deployment. The post-deployment criterion is particularly important. We are responsible for the quality of our outputs to a standard that our sub-processors—OpenAI, Anthropic, and others—are not. Their liability ends at the API. Ours does not.

To address this, we have divided our testing into categories, each with specific criteria to determine appropriate use cases.

Imperative Testing (Structured Output)

When: Pre-deployment

This is the most straightforward and most familiar method. It resembles traditional test suites.

The approach: structure prompts to deliver structured outputs. Maintain a vetted library of inputs with known expected outputs. Run the suite on each deployment candidate and compare results.

How it works in practice:

We maintain test cases derived from real clinical scenarios. Each case includes:

• Input data (patient records, procedure types, clinical context)

• Expected output structure (risk flags, missing items, recommendations)

• Validation criteria (which fields must match, which can vary, and which indicate failure)

For a prompt designed to identify medication contraindications, the test case might include a patient on Ozempic scheduled for a procedure requiring discontinuation. The expected output is a flag indicating that the Ozempic must be held. If the system fails to produce that flag, the test fails.

Benefits:

• Fast execution

• Low cost

• Deterministic pass/fail criteria

• Integrates with existing CI/CD pipelines

Limitations:

• Brittle. Non-deterministic outputs may fail tests despite being clinically valid.

• Limited coverage. Test cases cannot anticipate every real-world scenario.

• False confidence. Passing tests does not guarantee correctness on novel inputs.

We use imperative testing as a baseline. It catches blatant regressions and ensures structural consistency. It does not catch subtle degradations in clinical reasoning.

Hash Checks

When: Post-deployment

We use a testing method we call Hash Checks when validation is cheap, but determination is complex. In these scenarios, validating a correct solution is inexpensive and can be done alongside solutions produced by an LLM or other ML model.

How it works in practice:

We have multiple small systems that we use to detect show-stopper conditions that are often buried between multiple sources, but these conditions can be confirmed by the patient with a simple yes/no question. There are literally hundreds of these conditions we check in any given case, which makes asking all questions of each patient impractical and, counterintuitively, less accurate.

After detecting these conditions, we can confirm via a notification sent to the patient or their past provider. Using this two-step system, we reduce the cognitive overhead on the patient, remove steps from the process, and increase the coverage of conditions we’re searching for.

One drawback to this approach is that delivery is not guaranteed, which means you will need to design a failure state into your system. Delivery cannot be guaranteed in all scenarios because your imperative checks can disagree with the AI output (i.e. the LLM’s answer fails the static imperative checks and therefore is not surfaced to the user), creating a conflict.

Benefits:

• Highly accurate, complex results delivered to users

• Broader range of checks

Limitations:

• Solutions must be confirmable through deterministic means

• Delivery is not guaranteed

Human-in-the-Loop

When: Post-deployment

For sensitive scenarios, we engage a human to validate output before delivering the final result to the customer.

The approach depends on the risk level:

Sampled review: A percentage of outputs are routed to human reviewers. This provides ongoing monitoring of system performance without requiring review of every case.

Full review: All outputs for designated high-risk scenarios are routed to human reviewers before delivery. The system does not release the output until a reviewer approves it.

How it works in practice:

When Orchestra generates a readiness assessment for a high-acuity case—such as cardiac surgery, transplant, or complex oncologic resection—the output is retained. A clinician from our reviewer network receives the case, reviews the source data against the system’s output, and either approves, modifies, or rejects the assessment.

Approved assessments are delivered to the customer. Modified assessments are delivered with corrections. Rejected assessments are flagged for engineering review.

The review serves two purposes:

1. Validating sensitive outputs. The customer receives an assessment that has been verified by a qualified clinician, not just generated by a model.

2. Flagging faulty outputs. Rejected and modified assessments feed back into our monitoring systems. Patterns of modification indicate systematic issues. Rejections indicate potential model failures.

Benefits:

• Highest confidence for critical outputs

• Generates labeled data for model improvement

• Provides clinical validation our sub-processors cannot offer

Limitations:

• Slow. Human review adds latency.

• Implementation is expensive

We can deliver this human-in-the-loop validation at scale because of the clinical network we have built over the past year. This network — physicians, nurses, pharmacists, and other clinicians—provides the capacity to review outputs across specialties and time zones. It also provides the domain expertise to catch errors that a general-purpose reviewer would miss.

For our customers, this means they receive clinically validated patient readiness assessments without requiring their own staff to perform the validation. For us, it means we have a real-time signal on system performance from the people most qualified to evaluate it.

Outcome Tracking

When: Post-deployment (delayed)

The ultimate test of a patient readiness system is patient outcomes. Did the flagged risks materialize? Did the missed risks cause harm? Did the assessments improve decision-making?

The approach: track downstream outcomes and correlate them with system outputs.

How it works in practice:

For patients whose assessments flagged specific risks, we track whether those risks manifested. A patient flagged for cardiac risk who experiences a cardiac event during surgery validates the flag. A patient flagged for bleeding risk who proceeds without incident may indicate an overly sensitive threshold—or may indicate that the flag prompted appropriate precautions.

For patients whose assessments did not flag risks, we track adverse events. An unflagged patient who experiences a preventable complication indicates a miss.

Over time, these correlations inform threshold tuning, prompt refinement, and overall system calibration.

Benefits:

• Ground truth evaluation

• Directly measures clinical value

• Informs continuous improvement

Limitations:

• Long feedback loops. Outcomes may not be known for days, weeks, or months.

• Confounded by interventions. A flagged risk that does not materialize may mean the flag was wrong, or may mean the flag prompted actions that prevented the event.

• Requires data access. Not all customers share outcome data.

We use outcome tracking as the long-term signal. It does not catch issues in real time, but it tells us whether the system is doing what it is supposed to do: improving patient safety.

The Testing Stack

No single method is sufficient. Each addresses a different failure mode, operates on a different timescale, and provides a different type of signal.

Imperative Testing

Timing: Pre-deployment

Catches: Structural errors, obvious regressions

Misses: Subtle reasoning failures

Hash Checks

Timing: Post-deployment

Catches: Output drift, unexpected changes

Misses: Whether changes are good or bad

Humans-in-the-loop

Timing: Post-deployment

Catches: Clinical errors, edge cases

Misses: Scale limitations

Outcome Tracking

Timing: Delayed

Catches: Real-world effectiveness

Misses: Immediate issues

We run all of them. They are not redundant. They are complementary.

Testing in practice

The patient who died had a documented cardiac incident from five years prior. The documentation existed. It was in the record. A human reviewer with unlimited time could have found it.

The problem was not missing data. The problem was buried data. Five years of subsequent records—hundreds of notes, thousands of data points—had pushed the relevant signal below the threshold of human attention.

Our system surfaced it. It did so because we designed it to look for exactly this kind of risk, and because we tested it against cases where these risks were present and missed.

That customer now uses Orchestra. Their coordinators no longer rely on their own ability to read through five years of records and hope they catch the relevant detail. The system flags it. A human confirms it. The surgical team knows before the patient enters the OR.

Modern-day surgery and anesthesia care are nothing short of a modern-day medical miracle. Think about it. It’s now considered routine to receive incredibly complex and specialized care, with the expectation that your surgical problem is safely addressed while feeling little to no discomfort during the experience. Millions of surgeries happen in the US and globally every year, and the care teams that perform surgery are well-trained masters at what they do.

The current state of safe surgical care

However, despite the marvel that is modern surgical care, surgery itself can still be a messy and scary experience with thin margins for error. Complications do happen, and even with all of the advances in care over the decades, lapses in communication, missed diagnoses and poor care coordination can result in a scenario where the complexity of the care exceeds the capabilities of the health system. Recent studies have shown surgery complications of any cause range from 3-5%, and the risk of death or a major heart problem during or after surgery can be 2-12% depending on your health history. These rates are higher than many people are led to believe before embarking on their surgical journey. Can you imagine if there was a problem with 1 out of every 20 airplane flights?

Surgery is a cardinal event requiring focus and preparation

From the patient’s perspective, surgery is a rare event that deserves special attention and concierge care. On average, a person will only have surgery 2 or 3 times in their life. These are cardinal events that should receive as much attention and plan as having a child, getting married, or graduating from school. Having surgery is a critical time when you expect everyone to be at the top of their game, and the stakes are high for everything to go right.

Meanwhile, the time leading up to surgery has often been called a “black hole”, with poor communication and inadequate preparation being the norm. Over 1 in 5 outpatient surgeries are canceled the day of, mainly for logistical errors that could have been caught days in advance. So, how can we identify and prevent potential problems to achieve the best chance for success? Enter perioperative care.

What is perioperative care?

Perioperative, or periop, care means care surrounding surgery. It is a specialized field in medicine encompassing preop, intraop, and postop care with a focus on improving the preparation and recovery phases of surgery.

First coined in the 1970s, there has been little time or attention placed on coordinating surgical care end-to-end in the intervening years until recently. Anesthesiologists, the specialty most concerned with a patient’s condition leading up to and following surgery, have been at the forefront of this progress largely due to the ASA (stands for American Society of Anesthesiologists).

The idea is that if care teams and patients fully engage, they can devise a safe plan for surgery while preventing problems and complications. You wouldn’t take a car with a questionable maintenance history on a cross-country trip, and likewise, you should make sure your care team fully understands you and that your health is optimized before surgery. The problem is, health systems and many clinical teams aren’t well-positioned to perform perioperative care well.

How does it work?

Perioperative care requires coordination and medical guidance that takes a patient from wherever they are in their current state of health, and safely guides them through the surgical journey and back to, or hopefully better, than where they started. In our current fee-for-service health system, there is a strong financial incentive to focus on the surgery and anesthesia care itself, but there are wide gaps in the preparation and recovery phases of surgery. As a result, we are now routinely seeing surgical cases get canceled up to 20% of the time due to poor preparation, and hospital readmission rates and post-surgical infection rates are a stubborn cause of cost and harm. And yet, many doctors accept “barely good enough” as routine. Our attitude at Orchestra is that barely good enough is completely unacceptable.

‍

Who performs perioperative care?

High-quality perioperative care is difficult and expensive to implement because it is complex and no single clinical team or health system is well-positioned to deliver it. Primary care practitioners (PCP) are experts at delivering outpatient and chronic care but are often left out or unequipped to participate in the surgical care journey. Surgeons can work magic in the operating room, but typically lack the resources or incentives to provide high touchpoint preop and postop care. Anesthesiologists are trained to be at the intersection between medical care and surgery, but most anesthesia care teams have little interest in outpatient care coordination and would prefer to stay within the walls of the operating room. Specialists, such as cardiologists, often work completely separate from the surgery system and are unable to coordinate with surgical teams.

Finally, because perioperative care is a niche service, none of these teams are able or know how to bill for the effort. When there’s no financial incentive, the work doesn’t get done. Ultimately, the end result is care teams that do a poor job of or actively avoid perioperative care altogether.

Orchestra health is democratizing perioperative care

Orchestra is solving these problems and delivering consistent, high-quality perioperative care by guiding all stakeholders through the process from beginning to end, essentially providing the necessary structure for everyone to communicate effectively while allowing them to focus on their individual specialties. Orchestra’s care model communicates directly with surgeons’ offices, facilities, patients, and anesthesia care teams to solve communication and coordination issues. We provide more touchpoints and act as patient advocates to connect with outside teams like PCPs and specialists to make sure the broader health system is coordinated during surgical care. As experts in perioperative business management, we can bill for the medical and coordination service and are contracted with most major payors. Finally, we leverage the latest technology to engage directly with patients to ensure the highest clinical quality and patient experience.

Please contact us today to learn more about how Orchestra Health can drive top-line revenue while delivering best-in-class perioperative care to your surgical services.

Today, we're excited to announce the General Availability of Study Tracker on the Orchestra Health Platform. You should be able to find this on the surgery slide-out. At a high level, Study tracking will allow you to customize the necessary labs, studies, documents, and approvals needed to perform each surgery. Newly registered surgeries will begin blank.

This list is a collation of requirements defined by the surgeon, facility, patient profile, and surgery profile. As requirements begin populating you will begin to Orchestra automatically evaluate them against all thresholds.

Example valid Study

Example expired Observation

Example Panel with out of range requirements

Configuration

You will now also see an Icon on your toolbar for Study Tracking. This toolbar icon will allow you to continuously adjust and refine which requirements are needed within your organization, and provider. Any updates you make on this page, after saving, will immediately reflect all active surgeries.

This introduction of Study Tracker will allow us to better work together to understand your needs and requirements as we continue to prep upcoming patients.

Subscribe now