Visual Summary Dashboards

Explore CBE assessment data by learner, by assessor, by program and across programs.

This article covers the following topics:

• Introduction to Visual Summary
• Prerequisites
• Permissions
• Access Visual Summary
• Resident Dashboard
• Normative Assessment Dashboard
• Faculty Development Dashboard
• Program Evaluation Dashboard
• Program Oversight Dashboard

Introduction to Visual Summary

Visual Summary is an optional set of dashboards for organizations using the Competency-Based Education (CBE) module. It presents completed CBE assessments as charts and tables, so competency committees, program leadership and learners can review progress beyond what the CBE Program Dashboard shows.

Visual Summary includes five dashboards:

• Resident Dashboard: a detailed view of one learner's assessments, organized by training stage and EPA.
• Normative Assessment: each learner's progress compared with others in the program.
• Faculty Development: assessments organized by the assessor who completed them.
• Program Evaluation: program-wide assessment trends across academic years.
• Program Oversight: key metrics compared across several programs.

Visual Summary data is prepared by a scheduled process in Elentra rather than in real time. Recently completed assessments appear in the dashboards after the next scheduled update.

Note: In organizations whose name includes "Undergraduate" or "UG," the Resident Dashboard is labelled Learner Dashboard.

Prerequisites

Before using Visual Summary, an organization needs:

1. The CBE module enabled.
2. Visual Summary enabled by Elentra Support. It can be enabled for the whole organization or for individual courses or programs.
3. Assessment plans built for each course or program in the Assessment Plan Builder. Visual Summary uses assessment plans to calculate progress and achievement.

Permissions

Elentra shows each user only the Visual Summary dashboards that match their access.

Who can open Visual Summary from My Learners:

• Competency Committee members
• Academic Advisors
• Assessment & Evaluation administrators
• MedTech:Admin

Which dashboards each user sees:

Program Oversight compares only the programs a user can access. These are default permissions, and an organization's permissions may differ.

Access Visual Summary

Faculty and staff:

1. Click the Assessment & Evaluation icon at the top right.
2. Click the My Learners tab. The CBME Program Dashboard opens.
3. If you have access to more than one program, select the program from the course picker.
4. Click the Visual Summary tab below the filters.
5. Click a dashboard name to switch between dashboards.

Learners: Learners open Visual Summary from the Visual Summary tab on their own CBE dashboard. They see the Resident Dashboard with their own data only.

Resident Dashboard

The Resident Dashboard shows one learner's completed assessments in detail, to support competency committee review.

Select a learner: Choose a learner from the drop-down in the filter panel, then click Get Records. Type part of a name to filter the list. Clicking a learner on the Normative Assessment dashboard opens their Resident Dashboard directly.

Acquisition Metrics: Shows three totals for the learner, with a chart of their weekly EPA acquisition over the last six months.

• Total EPAs Observed: all EPAs completed on the learner, including those on expired or archived forms.
• Progress Rate: EPAs achieved, divided by the total number required in the assessment plan across all training stages.
• Achievement Rate: EPAs achieved, divided by EPAs completed. An achieved EPA meets the requirements set in the assessment plan, such as a minimum rating or specific contextual variables.

Recent EPAs: Plots the learner's most recent assessments by entrustment rating, oldest on the left. Choose the last 10, last 25, last month or last 3 months. Hover over a point for details. Assessments on different rating scales appear in separate charts.

Individual EPA Assessments: Groups every EPA by training stage. Each stage shows whether it is complete or its completion rate, and expands to list its EPAs. For each EPA, the dashboard shows:

• The EPA code and description.
• Observed, achieved and required counts, with progress bars. A green check mark appears when the learner has achieved the required number or the competency committee has marked the EPA complete. Otherwise, the number still to go is shown.
• A chart of every assessment on that EPA by date and rating. Hover over a point to see narrative feedback, context and the assessor.

Three icons below each chart:

• Filter (sliders icon): highlights assessments matching a contextual variable, such as Case Complexity or Clinical Presentation.
• Table (book icon): lists the assessments in a table that can be sorted and searched.
• Info icon: shows the achievement criteria for the EPA.

Expired assessments: If an organization tracks expired assessments, a table at the end of the dashboard lists assessments that expired before completion.

Date filter: Click the calendar icon in the filter panel and set a start and end date. Assessments in that period appear as diamonds across the dashboard. Select the checkbox to hide EPAs and stages with no assessments in that period. This is useful for reviewing activity since the last competency committee meeting.

Normative Assessment Dashboard

The Normative Assessment dashboard compares the EPA progress of every learner in a program, to give competency committees context for each learner's performance.

• Stage filter: View all learners, or only learners currently in one training stage.
• Learners with no records: Learners without completed EPAs are hidden by default. Clear the checkbox at the top of the dashboard to include them.
• Bar chart: Each bar is one learner. Use the options above the chart to switch between Total EPAs, Achievement Rate, Progress Rate, and Total EPAs vs Achieved EPAs. Comparing total and achieved EPAs side by side gives better context for learners with few completed EPAs.
• Table: Lists the same metrics for each learner. Click a column header to sort.

Click a learner's bar or table row to open their Resident Dashboard.

Faculty Development Dashboard

The Faculty Development dashboard organizes the program's completed EPAs by the assessor who completed them, to inform faculty development.

• Filters: Narrow the data to one assessor, one assessor group (for example, to include or exclude learner assessors) or one department.
• Acquisition metrics: Totals for all assessors in the selected academic year, or for one selected assessor, including the spread of entrustment ratings and training stages.
• Assessor charts: Each chart shows one bar per assessor. Click a bar to select that assessor.
  • EPAs observed.
  • EPA expiry rate: the share of EPAs sent to the assessor that expired before completion.
  • Average entrustment score.
  • Average words per comment.
• Assessor tables: For a selected assessor, tables list the EPAs they completed and the EPAs that expired before they completed them. The expired EPAs table can be exported as a CSV file.

Entrustment rating charts include only assessments completed on Supervisor Forms that use a standard 5-point O-Score scale.

Program Evaluation Dashboard

The Program Evaluation dashboard shows the program's completed EPAs across academic years, to inform program evaluation.

Across academic years:

• EPAs Acquired and Expired: EPAs completed and expired each year, in total and per learner.
• EPA Rating Distribution: The share of EPAs rated at each entrustment level each year. Includes only Supervisor Forms with a standard 5-point O-Score scale.
• Average Words per Comment: The average length of narrative feedback each year. Longer feedback has been found to correlate with higher-quality feedback.
• Monthly Distribution: EPAs completed in each month, July to June, to show seasonal patterns.

For a selected academic year: Summary metrics for the year, plus charts showing assessors by type, group and role. Group and role are available for internal assessors only.

Export Program Data: Downloads a CSV file of every assessment completed or expired in the program, for further analysis.

Program Oversight Dashboard

The Program Oversight dashboard compares key metrics across programs. It is available to users who can manage assessments in more than one program.

Select the programs to compare and an academic year, then review:

• EPAs Acquired and Expired: EPAs completed and expired each year, by program.
• EPAs Acquired and Expired per Learner: The same counts per learner, with a dotted line showing the average across the selected programs.
• Average Words per Comment: The average feedback length each year by program, with a dotted line showing the average across the selected programs.
• Monthly metrics: EPAs completed and expired each month of the academic year, for each program.