How well does Adapt predict Success: Part 1 of 3

Last year, some of our staff members attended the Actuarial Research Conference (ARC). They put together a presentation on Adapt. We wanted to share the research with all of you!

This is a 3-part blog series:

Blog 1: Background

Blog 2: Discussing Success Factor 1

Blog 3: Discussing Success Factors 2 & 3

Let us know your thoughts in the comments below.

Context

Every student wants to pass the actuarial exams. They know that in order to do so, they have to do tons of practice questions. There are many practice banks in the market that students could use. The problem is Screen Shot 2015-04-15 at 2.06.03 PMstudents are at different levels of readiness for the exam. If you evaluate the knowledge of students preparing for the exam on a scale of 1 to 10, some would be beginners, or close to 1. Others would be advanced, or close to 10. Many are somewhere in between. Unfortunately, most of the practice banks in the market are targeted toward students at a certain knowledge level. If you are not at that level, then you, the student, must adjust. If your knowledge is below that level, you may be discouraged by questions that are too difficult. If your knowledge is above that level, you are wasting valuable time on questions that are too easy.

Wouldn’t it be better for the practice questions to adjust to you, rather than for you to adjust to the practice questions?

Solution

Introducing Adapt. Adapt stands for Adaptive Dynamic Actuarial Practice Tests. It is an online testing software that adapts to a student’s current knowledge level. Adapt generates unique exams from a large pool of questions and allows students to work on questions that are neither too difficult nor too easy. Students begin working on problems at their own knowledge level. As questions are mastered, Adapt will provide new ones at a more difficult level. The goal is for students to attempt questions that are at exam-level difficulty before taking the actual exam.

Screen Shot 2015-04-15 at 2.07.29 PMHow Is Adapt Engineered?

Adapt uses an indicator called the “earned level” to measure the progress of Adapt users. The earned level ranges from 0 to 10. Every Adapt user starts with an earned level of 3. After the user completes a practice exam, the earned level will change. If the user performs well on the practice exam, the user’s earned level will increase. As a result, the user will be exposed to more difficult questions. If the user performs poorly on the practice exam, the user’s earned level will decrease. As a result, the user will be exposed to easier questions. This will help to solidify the user’s understanding of fundamental concepts.

The real magic of Adapt is combining the user’s earned level with the difficulty level of practice problems. Each practice problem is assigned a difficulty level ranging from 0 to 10. If your earned level is 4, Adapt will give you mostly level 3 to level 5 questions. This swing introduces an element of variability that is present on the actual exam. This swing also gives you a set of questions which provides some review and prepares you for the next level.

The bottom line is that because the user is getting the right questions at the right time, the user is challenged but not overwhelmed. Not only will users be able to track their progress through earned levels, they will be more motivated as their earned level increases. We believe that is another step toward success.

Screen Shot 2015-04-15 at 2.08.55 PM

We recommend getting at least an earned level of 7 to be exam ready. Why? This question will be answered in parts 2 and 3 of this blog series.