Inside Story: Sakai Annual Upgrade and Quality Assurance

We are excited to introduce a new version of Sakai each summer. The annual update brings a variety of new features alongside hundreds of bug fixes, all in the hopes of making Sakai more accessible, useful, and easy to understand. While users explore the new features and familiarize themselves with this summer's update, we’d like to offer insight into the version upgrade process and the Quality Assurance (QA) effort.

Each Claremont Sakai upgrade consists of several phases as well as the collaboration of many consortium members. Upgrades start with the selection of the next stable Sakai version. After a version is selected, related system requirements are evaluated. Then a target timeline is determined, and the systems are designed, along with implementing local changes and third-party tools. Once the system is in place, the final phase before project rollout is testing.

While all phases are important for a successful upgrade, ensuring Sakai Quality Assurance (QA) is our highest priority. Within the QA process, tool and feature testing are among the most crucial steps. 

The Sakai QA team consists of various testers from all Claremont colleges, each school’s Sakai admins, and a consortial administrator. The QA testers have different uses and priorities for Sakai, and they have different levels of Sakai knowledge. The Sakai QA hub is created to cultivate a collaborative testing environment by:

  • Improving the reliability of testing results
  • Avoiding any overlapping in testing efforts
  • Providing up-to-date test results to the members
  • Facilitating testing progress management and monitoring

The Sakai QA team tries to ensure that Sakai works as intended for everyone by quickly identifying and correcting any errors they observe with the platform. To improve accessibility, each feature is tested with multiple browsers and devices from both a student and faculty perspective. Testers begin by going through essential tool functions to ensure there aren’t any glaring issues. Testers then evaluate whether new features have been adequately integrated into the platform and ensure that those features don’t disrupt other functions. Finally, testers try to reproduce bugs to see if they’ve been fixed or if there are still ongoing problems. If an issue is identified, testers report the bug to a consortial administrator. Once the issue is resolved by a consortial administrator, it gets incorporated into the testing process.

A predefined and online Test Template is compiled and shared with the Sakai QA team for collaborative testing. It includes a list of test plans, test cases with step-by-step instructions, test status, devices and browsers, and a record of when issues were reported/resolved/etc. This template is also used for confirmation when issues are encountered after an upgrade. 

A well-prepared test plan and predefined templates establish a robust testing environment where all team members know what to expect from the beginning and how to collaborate throughout the QA process efficiently. A formal, well-established, and collaborative testing environment enforces better testing conditions and helps the QA team get efficient testing results within an allotted time frame. This testing environment also guides the team toward what should be done better next time with self-presenting proof.

While the Sakai QA team is dedicated to improving Sakai, it is worth mentioning the role of user feedback in our QA process. Many of the bugs tested are user-reported, and many of the features added are user requested. In a way, every member of the Claremont Colleges is like a member of the QA team.

When combined, the Sakai QA team tested over a thousand features, bugs, and newly added features this year. The consortium achieved this number of tests thanks to the work of the Sakai QA team and the engaged faculty and student body. We will continue to do our best work, but we also want to share our appreciation for those members of the Claremont Colleges who have offered feedback. Moving forward, we hope to work closely and symbiotically with students and faculty alike to ensure that all our users can enjoy a version of the learning management system that suits their needs.