DoD FY15 Metrics Transitioning Training to Reality (RealMETRX) Award | Apply Online

Opportunity Summary:

The DoD FY15 Metrics Transitioning Training to Reality (RealMETRX) Award (Funding Opportunity Number W81XWH-15-DMRDP-MSIS-REALMETRX) was a Department of the Army/USAMRAA research funding opportunity under CFDA 12.420 (Military Medical Research and Development) aimed at solving a very practical problem in military and civilian medicine: simulation-based training is widely used, but the field still struggles to prove, with objective evidence, which simulation-measurable indicators actually predict better real-world patient care. RealMETRX sought projects that would determine, define, and validate training proficiency metrics and evaluation criteria that can be captured in medical simulation systems and that are empirically linked to optimal clinical performance and patient outcomes. The program emphasized acute trauma care, critical care, and prolonged care, and encouraged inclusion of military-relevant injuries while still producing metrics broad enough to generalize across military, VA, academic centers, rural settings, public/private hospitals, and even international contexts. The funding mechanism was a grant/cooperative agreement, with an estimated total of $3.2M and about 2 expected awards, no cost sharing required, and eligibility described as unrestricted. A core expectation was that awardees would use rigorous statistical approaches to identify the best-performing metrics and show that they track the transition from simulation training to actual practice across as much of the continuum of care as possible (from first encounter through discharge and near-term follow-up).

For measuring effective decision making (for novices and more experienced clinicians), the strongest approach is usually not a single score but a structured set of metrics that capture (1) the process of decision making under uncertainty, (2) the timeliness and appropriateness of actions taken, (3) teamwork and communication behaviors that directly influence execution, and (4) patient-centered outcomes and safety signals that are plausibly sensitive to training. In trauma and critical care, good decision making is often reflected in whether the clinician recognizes key patterns early, selects the right priorities, sequences actions correctly, adapts to changing physiology, and avoids preventable harm. That can be measured in simulation through time-stamped event logs, physiologic trajectories produced by the simulator, eye tracking or attention proxies when available, structured rater instruments, and analysis of the decisions chosen at critical branching points. In addition, because RealMETRX emphasizes connection to real care, the same constructs should be mapped to clinical data sources (EHR timestamps, order sets, vitals trends, complications, and follow-up events) to test whether simulation performance predicts actual performance after controlling for case-mix and context.

The most defensible decision-making metrics tend to fall into several categories. First are time-to-critical-action measures that reflect prioritization and recognition, such as time to hemorrhage control, time to activation of massive transfusion protocols, time to antibiotic administration in suspected sepsis, time to airway stabilization when indicated, time to lactate measurement and reassessment, and time to escalation (calling for help, consult, transfer to higher level of care). These are attractive because they are objective, timestampable in both simulation and clinical practice, and often associated with outcomes. Second are guideline- and evidence-concordance measures, where the evaluation checks whether actions match validated pathways (ATLS/trauma resuscitation principles, ventilator lung-protective strategies, sepsis bundles, DVT prophylaxis decisions, sedation/analgesia best practices, transfusion ratios, damage control resuscitation, and appropriate imaging choices). Third are error and near-miss measures, which can include omission errors (failure to reassess, failure to recognize shock, failure to consider reversible causes), commission errors (unnecessary intubation, inappropriate fluid choice, contraindicated medication), and safety-critical violations (wrong dose, wrong route, contamination breaks, failure to confirm tube placement). Fourth are adaptive decision-making measures that look at whether the clinician updates the plan when new information arrives (dynamic re-triage, changing differential diagnosis, revising ventilator settings when compliance changes, or stopping a harmful intervention promptly). In simulation, this can be captured as “decision pivots” and the latency between a physiologic change and the corrective action.

To better reflect the multitude of variables across the continuum of care from initial encounter to discharge and 6-month follow-up, the best metric strategy is to build a chain of linked indicators rather than jumping straight to long-term outcomes alone. Long-term outcomes are important, but they are heavily confounded by injury severity, comorbidities, resources, and post-discharge social factors. Training-sensitive indicators can be built as intermediate outcomes and process reliability measures that are more directly attributable to decision quality. Examples include stabilization quality (shock index improvement, lactate clearance, avoidance of hypoxemia/hypotension episodes, timely temperature management), complication prevention (ventilator-associated events, catheter-associated infections, pressure injuries, delirium burden, DVT/PE prevention adherence), and discharge readiness measures (med reconciliation quality, follow-up scheduling, return precautions documented and communicated, early outpatient complication detection plans). Near-term follow-up metrics within 6 months could include unplanned readmissions, ED revisits, preventable complications, functional outcomes when feasible (return to duty/work, mobility, neurocognitive function after TBI), and patient-reported outcomes. The key is to treat these as downstream validation targets while still focusing training measurement on upstream decisions and behaviors that plausibly drive them.

A particularly powerful set of evaluation criteria for decision making involves measuring situation awareness, clinical reasoning, and cognitive workload in ways that can be operationalized. Situation awareness can be approximated with “information gathering completeness and relevance” (what data were sought, in what order, and whether irrelevant testing was minimized), “anticipation and planning” (whether the clinician prepared for deterioration, pre-briefed the team, staged equipment, or ordered blood early), and “reassessment discipline” (frequency and appropriateness of reassessments after interventions). Clinical reasoning quality can be assessed by scoring differential diagnosis breadth early vs late, appropriateness of hypothesis updates, and consistency between stated diagnosis and actions (the plan matches the claimed mental model). Cognitive workload and stress resilience can be measured via performance under time pressure (decision latency, error rate under load), secondary-task performance in simulation, or physiologic proxies if available; even without advanced sensors, variability in decision times and escalation patterns under increasing scenario complexity can serve as an indirect workload marker.

Because RealMETRX explicitly wants metrics that can accelerate maturity and experience, the most useful evaluation criteria are those that show clear learning curves and discriminate between novice, intermediate, and expert performers. Commonly, experts distinguish themselves not just by speed, but by selecting the right actions with fewer unnecessary steps, maintaining broader awareness, and preventing errors. Metrics that often show this include efficiency measures (task completion with fewer non-value-added actions, fewer redundant orders, fewer cancellations), prioritization accuracy (correct ordering of interventions), and “stability of performance” (less variance across scenarios, better performance in rare but critical events). Another expert-like hallmark is calibrated confidence: knowing when to act decisively and when to pause and gather more data. This can be measured by comparing confidence ratings to correctness (calibration curves), or by examining inappropriate certainty (aggressive irreversible actions taken with weak evidence) versus appropriate uncertainty management (seeking confirmatory signs, escalating to expert consultation). In team-based trauma and ICU care, maturity also shows up in closed-loop communication, role clarity, effective delegation, and early activation of resources; these can be rated with validated teamwork frameworks and also captured with objective speech-event coding in recorded simulations.

For comparing novices and not-so-novice personnel to high-performing experienced colleagues, the cleanest method is to define benchmark distributions based on expert performance and then compute gap-to-benchmark scores across multiple dimensions. That can include z-scores against expert medians for time-to-action, error rates, adherence measures, and physiologic outcome proxies within the simulation (time in hypotension, time in hypoxemia, total “harm exposure” minutes). A useful framing is “entrustment”: how much supervision a clinician would require for a given scenario. Entrustable Professional Activities (EPAs) or entrustment scales can be adapted to acute care tasks (leading a trauma resuscitation, managing ventilated shock, initiating massive transfusion, coordinating prolonged field care), with anchors that translate to observable behaviors. Pairing entrustment ratings with objective logs helps reduce subjectivity. Another strong approach is generalizability theory or many-facet Rasch modeling to account for rater severity, scenario difficulty, and learner ability, enabling fairer comparisons across different cases and evaluators.

On the statistical side, the opportunity implicitly pushes teams to move beyond simple checklists and toward validated measurement models. That typically means identifying candidate metrics, testing reliability (inter-rater reliability for observational scores; test-retest reliability; internal consistency where relevant), demonstrating construct validity (experts outperform novices; performance improves with training), and then demonstrating criterion validity (simulation metrics predict clinical performance or outcomes). Because real outcomes are confounded, the best analyses often use multilevel modeling (patients nested within clinicians nested within sites), risk adjustment (injury severity scores, comorbidity indices), and propensity or matching techniques when comparing trained vs untrained cohorts. For selecting the “best” indicators among many candidates, approaches like regularized regression (LASSO/elastic net), random forests/gradient boosting with careful cross-validation, and interpretable models that allow clinical insight are common. A practical deliverable is a small set of high-yield composite indices (for example, a Trauma Decision Quality Index) built from weighted components that remain interpretable and can be computed in both simulation and clinical datasets.

Within acute trauma care, high-value metrics often include early recognition and control of life threats (airway compromise, tension pneumothorax, major hemorrhage), correct triage decisions (appropriate imaging vs OR, transfer timing), and avoidance of secondary insults (hypotension and hypoxia in TBI, hypothermia in hemorrhage). Simulation can measure the “burden” of these insults as area-under-the-curve time below thresholds, plus the latency to correction. In critical care, decision metrics that map to outcomes include ventilator strategy choices (tidal volume per predicted body weight, plateau pressure awareness, PEEP titration rationale), shock management patterns (fluid responsiveness assessment, timely vasopressor initiation, source control decisions), and sedation/analgesia/delirium practices. In prolonged care, especially in austere or resource-limited environments, metrics shift toward planning, resource stewardship, complication prevention over time, handoffs, documentation quality, and contingency management. Decision making here is less about single rapid actions and more about maintaining patient stability, anticipating deterioration, and balancing risk when definitive care is delayed.

If the goal is to accelerate acquisition of maturity, the most actionable metrics are those that can drive feedback and deliberate practice. Metrics should therefore be (1) specific enough to coach (for example, “reassessed BP within 3 minutes after intervention” rather than a generic “good resuscitation”), (2) sensitive to improvement, and (3) linked to patient safety. Dashboards that show a learner’s trajectory over repeated scenarios, highlight their most frequent decision bottlenecks (late escalation, missed reassessment, inappropriate fixation), and compare them to expert patterns can shorten the path to competence. Including scenario complexity scaling is also important: learners should be measured across routine cases and edge cases where cognitive traps are common, because that is where expert decision making diverges most clearly.

Finally, RealMETRX’s emphasis on “training sensitive outcome indicators versus those currently used to evaluate tasks, skills, and procedures” suggests a shift from counting whether someone performed a procedure to measuring whether the overall care plan was correct, timely, coordinated, and safe. The best metrics for that are often hybrid measures: objective timestamps and physiologic impact paired with structured assessments of reasoning and teamwork. Done well, the result is a set of validated indicators that can be captured in simulation, that discriminate levels of expertise, that improve with targeted training, and that meaningfully predict better clinical care from first contact through discharge and near-term follow-up.

Browse more business programs for Training Services

Browse more opportunities from the same agency: Dept. of the Army USAMRAA

Browse more opportunities from the same category: Science and Technology and other Research and Development

Next opportunity: DoD FY15 Medical Decision Aids Predictive Markers (SimMarkers) Award

Previous opportunity: BLM Idaho Noxious Weed Control Program

USGrants.org Applicant Portal:

Are you interested in learning about about how to apply for this government funding opportunity? You can create a free applicant account and receive instant access to our applicant portal that many business owners like you have benefited from.

Apply for W81XWH 15 DMRDP MSIS REALMETRX

 

[Watch] how do funding administrators access proposals?

Applicants also applied for:

Applicants who have applied for this opportunity (W81XWH 15 DMRDP MSIS REALMETRX) also looked into and applied for these:

 

Grant application guides and resources

It is always free to apply for government grants. However the process may be very complex depending on the funding opportunity you are applying for. Let us help you!

Apply for Grants

 

 

Premium leads for funding administrators, grant writers, and loan issuers

Thousands of people visit our website for their funding needs every day. When a user creates a grant proposal and files for submission, we pass the information on to funding administrators, grant writers, and government loan issuers.

If you manage government grant programs, provide grant writing services, or issue personal or government loans, we can help you reach your audience.

Subscribe to Leads

 

Request more information:

Would you like to learn more about this funding opportunity, similar opportunities to "W81XWH 15 DMRDP MSIS REALMETRX", eligibility, application service, and/or application tips? Submit an inquiry below:

Don't forget to subscribe to our grant alerts mailing list to receive weekly alerts on new and updated grant funding opportunities like this one in your email.