How we review

If it’s your first time here – or you’d like a reminder – please take a look below. You’ll see both the steps we take and the things we look at when reviewing courseware products. Every product is examined by at least two members of the independent Product Advisory Board. Reviews are established through an evaluation process applied to each courseware product.

1. Identify

The CourseGateway team tracks the universe of digital learning tools to review and proactively solicits supplier participation.

2. Review

Courseware is evaluated by Product Advisory Board members and additional product users, leveraging the latest review process and methodologies.

3. Engage

CourseGateway actively collaborates with courseware suppliers to establish relationships and promote continuous improvement in the evaluated products.

4. Publish

Product Advisory Board members are given complete discretion to revise reviews based on the suppliers' feedback. Final reviews are published on CourseGateway.org.

Review Categories

Accessibility
Accessibility

Robustly accessible functionality, meeting technical standards (e.g. WCAG), with up-to-date and detailed conformance documentation, and exhibits mature practices, processes, and commitments.

View the rubric

The digital learning product has robustly accessible functionality and follows best practices that align with technical standards (e.g. WCAG).

The vendor provides current, complete, trustworthy, and informative documentation on the accessibility of the specific product.

The product’s supplier indicates maturity to consistently deliver accessible learning materials, environments, or platforms.

Efficacy

Existence and quality of research that supports a product’s effectiveness in improving students’ learning outcomes.

View the rubric

Supplier-generated case studies and qualitative studies are available across a variety of use cases and contexts.

Research is conducted across a diversity of institutional settings, and outcomes data is disaggregated across those institutional setting types.

Suppliers’ research designs and supports are available for institutions interested in pursuing research based on product-generated data.

Available research supports a product’s efficacy and leverages reliable and valid empirical methods.

Research has been conducted by a trusted third party (e.g., a nationally recognized research organization or a qualified researcher whose work is peer-reviewed).

Equity

Existence and quality of product attributes that promote equitable outcomes for Black, Latino/Latina, Indigenous students, and students from low-income backgrounds.

View the rubric

Research used to inform the design of the digital learning product is transparent and considers student populations by race and income.

The digital learning product uses equity-minded language.

Learning activities can be customized to embed culturally responsive and affirming functionalities.

Research is transparent and includes student outcomes evidence by race and income (e.g., case studies, quasi-experimental studies, experimental studies).

Students are consistently included and positively reflected.

The digital learning tool centers social, emotional, and academic supports that are culturally relevant and responsive, providing an affirming sense of belonging.

Functionality

Existence and quality of product attributes that relate to instructional design and user experience.

View the rubric

The tool adjusts the presentations of content in relation to the knowledge of learners.

Students can engage with peers, mentors, or instructors as appropriate within the tool’s interface.

Instructors or course designers can alter the learning or assessment content.

The tool supports evidence-based teaching pedagogies such as active learning.

Includes support to help learners grow beyond their current level of proficiency.

Learning can be assessed in relation to learning objectives/competencies.

Software and user-centered design features support sustained engagement and high user satisfaction (across both single sittings and the entire course).

Systems Capabilities
Systems

Technical components that indicate systems compatibility, ease of implementation, and data management capabilities.

View the rubric

Easy-to-use and robust functionality allows users to create, store, and share course-related content in various formats.

Scheduling, progress support, homework and assessment assignments, student-to-student communication, and student-to-instructor communication are supported functions.

Data generated and stored by the supplier is easily available to relevant faculty and administrators upon their request.

Applies an equity-centered lens and mindset to ensure data is collected, analyzed, interpreted, and shared with diverse stakeholders without bias or exclusion.

Students maintain ownership and copyright of their intellectual property/data; students can keep data private and decide both if and how their data is to be shared.

Incorporation of third-party applications is limited, and if necessary, does not inhibit student digital learning and product use.