This completes my Math Academy education in single-variable calculus, with Calculus I and Calculus II supplementing the material I learned in Mathematical Foundations II and Mathematical Foundations III.

As I usually state: Not everyone will share my opinions on Math Academy, and not everyone will want to use the Math Academy system as I do. But my comments may be useful or interesting to at least some people.

Skipping school

According to my activity log, I averaged just over 35 XP per day, with a lot of variability (standard deviation of over 25). I didn’t do any work at all on 18 days and did less than 10 XP on another 4; that’s almost 2 days a week when I skipped out on doing Math Academy. On the positive side, out of the 22 quizzes I took, I had to re-take only 12—far from perfect, but much better than what I was able to do in Calculus I.

Between “skipping school,” covering more new material, and a lot of reviews of material from previous classes (see below), it took me over three months (97 days) to finish Calculus II, considerably longer than I hoped when I began the course.

Just the facts, ma’am

I noted in my last update that I was starting to use Anki to try to remember the host of formulas related to trigonometric and hyperbolic identities, derivatives, and integral. Alas, this was not as successful as I’d hoped it would be.

The primary problem is that I have yet been able to make Anki reviews a daily habit: I’ll remember that I haven’t reviewed my cards in a while, do a session, and then forget to do it the next few days.

This problem is in turn probably due to two other factors: First, the sheer number of cards shown as needing to be reviewed is daunting, although most of them are for decks I don’t care about. Second, and I think more important, is that my Anki practice is disconnected from my Math Academy activity.

That’s why (once again) I’d like to see a Math Academy course focused on more advanced mathematical facts, as opposed to the kind of “math facts” course that’s been mooted, one intended for students learning multiplication tables and other common prerequisites. I think the chances of seeing such a more advanced course are relatively small in the next two or three years, but perhaps the Math Academy folks will have time to consider it at some point.

“Review hell”

A common complaint by people on the Math Academy discord server is that the number of reviews they have to get through is overwhelming, especially toward the end of courses. “Review hell,” some call it, frustrated that they aren’t seeing many new lessons compared to reviews.

I can put some numbers to that frustration: in the course of taking Calculus II, I had 87 lessons for Calculus II compared to 234 reviews of previous material (both in Calculus II and previous courses), about a 2.7-to-1 ratio of reviews to lessons.

I can also understand why people are frustrated: it’s annoying to be told that you’ve almost completed a course only to see review after review and only one new lesson. However, I also think that having lots of reviews is just a natural consequence of Math Academy having enabled you to complete lots of courses in a relatively short time.

Consider: in less than a year I completed 5 Math Academy courses, including a re-do of a comprehensive high school mathematics curriculum, the equivalent of a complete “Linear Algebra I” university course, and a fair amount of material from university-level courses on multivariable calculus and probability and statistics.

Of course I have a lot of material to review, how could it be otherwise? And I want to review that material; otherwise I’ll forget it all and the time (and money!) I spent learning it will have been wasted. So, you won’t catch me complaining about “review hell.” While I certainly don’t welcome seeing review after review appear in my queue, I recognize that they’re an important part of the Math Academy experience and I feel a sense of satisfaction when I can finish one without errors, knowing that I just helped solidify my knowledge of the subject.

Slowing down, taking a break

A major reason for my slowness in finishing Calculus II (I had originally planned to finish by the end of 2025) is that I’m working on a separate (and unrelated) project that’s taking up a lot of time. I’ll be doing that at least through the end of February, so I’ve decided to cut my daily XP goal from 40 down to 30 for now. I suspect I may also end up skipping a few days.

However, at some point I should have more time (perhaps much more time) and I’ll be ready to re-engage with Math Academy more fully. As I’ve previously mentioned, I plan to next take Multivariable Calculus and then Probability and Statistics, with a goal of finishing both this year. I have no idea what new Math Academy courses will be available after that, but I’m sure I’ll find something suitable.

That’s it for now, my next update probably won’t be for a few months.

My progress thus far

Here’s where things stand after one year: First, I’ve completed the following courses:

• Mathematical Foundations II
• Mathematical Foundations III
• Mathematics for Machine Learning
• Linear Algebra
• Calculus I

I’m also 91% of the way through Calculus II. I had hoped to combine this post with a post celebrating my completion of Calculus II, but unfortunately I got sick in late November and ended up skipping several days of Math Academy work in December. I should finish up Calculus II later this month.

I’ve accumulated a total of over 19,500 XP, averaging about 53 XP per day over the course of the year. If you ascribe to the Math Academy rle of thumb equating 1 XP to 1 minute of focused work, then I’ve spent about 325 hours working on Math Academy lessons, reviews, and quizzes, the equivalent of two months of a full-time job.

Why I’m doing this

A few weeks ago someone on X or on Discord asked me what my ultimate goal was in doing Math Academy. My initial interest, as I’ve written, was prompted by my failure to study linear algebra to a point where I knew what eigenvalues and eigenvectors were. But that wasn’t my ultimate goal.

The reason I started trying to learn linear algebra (way back when) was as an adjunct to learning probability and statistics. And that in turn was motivated by my desire to get to a point where I could read social science papers and understand the techniques being used. I’ve always enjoyed writing blog posts about various topics for which academic literatures exist, and I’ve always been frustrated by having to skim over the sections in papers where they discuss things like study designs, statistics methodology, causal inference, and the like.

So my goal is to be able to get to a point where I can go through books like Bayesian Data Analysis or Statistical Rethinking, follow the math and the R code, and maybe even try out some of the techniques on things I’m interested in for which I can get data.¹

A related goal is to improve my R and Python knowledge. I can do exploratory data analysis in R (including using ggplot2) and can do Python coding to a low intermediate level. I’d like to skill up to a more advanced level in both languages.² I‘m really looking forward to Math Academy adding a computer science course, but as noted below I’m not expecting to see it any time soon.

Math Academy then and now

As noted in my very first Math Academy post, I started my Math Academy journey not by enrolling in the service but by reading Justin Skycak’s book-in-progress The Math Academy Way and taking copious notes. That reading convinced me that Math Academy had what looked to be some reasonably sound learning science behind the service, and was therefore worth trying out.

The current version of the The Math Academy Way is about 12% more in length than the version I read originally, and is still not finished: There are five chapters marked as “in progress,” and 12 pages of “notes for future additions.” I suspect it will be some time before we see a complete first edition of the book.

The same can be said of the Math Academy service itself. The service is still described as having “beta” status, with no clear indication as to when it might shed that label. As with Gmail (which was labeled as “beta” for 5 years even as Google onboarded tens of millions of users), it’s possible that Math Academy won’t reach 1.0 status for a while.

That’s really just a minor semantic issue. The Math Academy service has been reasonably stable for the year I’ve been using it. The only thing out of the ordinary I’ve noticed is that the system takes a relatively long time to respond (several seconds at least) when you complete a lesson or a review. This may indicate increased system load due to more people using the service. Alternatively, it may (or may also) reflect the fact that choosing the next action may be increasingly time-consuming for students like me who‘ve already traversed a significant portion of the overall Math Academy knowledge graph. (In particular, I can imagine that deciding what topics have been reviewed thus far, including implicit review, may take a while.)

To my recollection, the only major product feature that’s been added during the past year is the “gravity” feature, which allows students to designate topics that they want to focus their learning toward. Math Academy then prioritizes lessons on the paths in the knowledge graph leading to those topics. Gravity is a beta within a beta, as you have to request access to it. I haven’t used the feature myself, because my interest is in completing entire courses, but the feedback I’ve seen on X and Discord is generally positive.

Most of the effort of the Math Academy team seems to be going toward creating new courses and making the backend and UI changes that will be necessary for at least some of those courses. To the best of my knowledge, the only completely new course added in 2025 was Discrete Mathematics, a university-level course.³ However, there were multiple updates to existing courses, including the addition of many free response questions, for which the student needs to type in an answer instead of selecting from multiple choices.

Several other courses were mooted as potentially appearing in 2025, including Differential Equations, Machine Learning (proposed as a two-part course), Computer Science (also proposed as a two-part course), and Abstract Algebra in the university-level courses, and SAT Test Prep and ACT Test Prep in the high school courses. Of those, the only one that appears likely to be released in the near future is Differential Equations.

There were also significant updates and expansions planned for SAT Math Fundamentals and Mathematics for Machine Learning. To my knowledge the Mathematics for Machine Learning expansion has not yet been released; that will presumably be done in conjunction with release of the first Machine Learning course. As for SAT Math Fundamentals, per Justin Skycak on Discord, “We added about 115 new ‘missing middle’ topics that appear on the SAT but are not covered in any standard school curriculum. For scale, 115 topics is roughly half the size of a full-year course like Algebra 1.”

The Math Academy team remains relatively lean. The only named individuals known to be employees are Jason Roberts (UI and backend development), Justin Skycak (analytics and algorithms), Alex Smith (content development), and Sandy Roberts (technical support and general administration). There’s also a group of people (presumably contractors) creating lesson content; their number and names are unknown.⁴

As discussed on one of Jason Robert’s and Justin Skycak’s podcasts, they don’t follow a set schedule for development, but rather concentrate their development efforts on whatever part of the product seems most urgent to focus on at the time. As a result there is no fixed roadmap for future courses, and I think it’s fruitless to speculate on what might be released when.

Speaking of podcasts, there are now a whole series of Math Academy podcasts featuring Jason Roberts, Justin Skycak, and (occasionally) Alex Smith. Some of them have interesting insights about the origins of Math Academy and its internal workings, but they’re pretty long and often rambling; making transcripts of them and having an LLM summarize the transcripts might be the best way to approach them.

Beyond Math Academy

As a result of using Math Academy, and especially of reading The Math Academy Way, I picked up a few other interests in 2025.

The first is PhysicsGraph, an online education service inspired by and modeled on Math Academy. I wrote an earlier post about the PhysicsGraph quantum computing course; I plan to write another post about their Physics I course. The PhysicsGraph folks are being much more aggressive than Math Academy in introducing new features in the UI and elsewhere; whether Math Academy will be inspired to adopt some of them is an open question.

Since I was following folks from Math Academy, PhysicsGraph, and related ventures on social media, I also stumbled upon the early reviews and articles about Alpha School. As it happens, Alpha School uses Math Academy as part of its educational software stack, thus highlighting another aspect of the Math Academy business model, i.e., licensing the service to schools instead of just serving individual subscribers. I wrote about the first podcast by Joe Liemandt, the funder of Alpha School and related ventures, and plan to write more about Alpha School in future.⁵

Finally, all this got me interested in developments in learning science, and so I started following Carl Hendrick, an academic working in the field who’s had some involvement with Alpha School. He has a newsletter that I’m subscribed to, and is also co-author or co-editor of several books on teaching and educational psychology, at least one of which I plan to read this year.

That’s it for now, I hope to be back here with another update in a few weeks once I finish Calculus II.

I should have already covered most of the material in Calculus I in prior courses, most notably Mathematical Foundations II and Mathematical Foundations III. However, I felt I had some significant holes in my knowledge, especially around hyperbolic functions (see below). So I wanted to make sure that I had complete coverage of single-variable calculus, especially before going on to more study of multi-variable calculus and other topics.

As I usually state: Not everyone will share my opinions on Math Academy, and not everyone will want to use the Math Academy system as I do. But my comments may be useful or interesting to at least some people.

Back on pace

After not quite meeting my 40 XP/day goal during the Linear Algebra course, I got back on track during Calculus I. According to my activity log, I averaged just under 42 XP per day. There were only 4 days when I did less than 30 XP, and just 1 day when I did no Math Academy work at all. However, I didn’t do so great on quizzes: I took 7 quizzes and did retakes on 6 of them.

I spent just under a month (28 days) taking Calculus I, as with Linear Algebra, a relatively short time that’s due to already having covered a lot of the material in the Mathematical Foundations courses.

Anki on the side

While working through Calculus I, I found myself struggling to remember all of the formulas that might come up in problems. This was especially true when I started the topics on hyperbolic functions, which introduce an entirely new set of identities and derivatives, similar to those associated with trigonometric functions but just different enough to cause confusion. I concluded that unless I could achieve better automaticity with these identities and formulas I was going to have a tough time going forward into multivariable calculus and other courses.

Unfortunately, Math Academy’s spaced repetition process wasn’t giving me enough review of standad formulas. Math Academy staff have teased a potential future “math facts” class; however it appears to be some ways off, and in any case sounds like it will focus more on elementary school material like multiplication tables.

It was therefore back to the old school spaced repetition solution for me, namely creating flash cards in the Anki app. (I wrote “app” singular, but the process is more complicated than that: I create cards using the Anki desktop app for macOS, synchronize them up to the Anki website, and then synchronize them down to the Anki iOS app on my iPhone.) Thus far I’ve added over 80 cards, from formulas for the volumes of spheres and other solids to the derivatives of inverse reciprocal hyperbolic functions.

Doing additional spaced repetition using Anki has helped a lot with simpler things like formulas for volumes and surface areas. However, I’m still having trouble with trigonometric and hyperbolic identities and formulas, where it’s easy to get lost in a maze of twisty little equations, all somewhat alike. I’ll need to continue working on this.

On to Calculus II

After finishing Calculus I, I’m immediately starting Calculus II. Per the Math Academy status information, I’ve already completed about 50% of its topics, so I should be able to finish up Calculus II well before the end of the year and conclude my study of single-variable calculus. Then it’s on to Multivariable Calculus and then Probability and Statistics. After that, who knows? I’d like to level up in my Python knowledge, so maybe I’ll look into the computer science course discussed on the recent Math Academy podcast by Jason Roberts and Justin Skycak.

That’s it for now, I hope to be back here with another update in a few weeks.

I’ve completed my fourth Math Academy course, Linear Algebra. As has become a habit by now, I’m celebrating by posting another update on my Math Academy experience. (For my experiences with my first course, Mathematical Foundations II, see my original series of posts summarizing the book The Math Academy Way and reviewing MFII. For my experiences with the Mathematical Foundations III and Mathematics for Machine Learning course, see my first and second updates.)

My usual disclaimer applies: Not everyone will share my opinions on Math Academy, and not everyone will want to use the Math Academy system as I do. But my comments may be useful or interesting to at least some people.

(Not quite) keeping up the pace

During the Linear Algebra course I didn’t quite maintain the daily goal of 40 XP that I set while taking the Mathematics for Machine Learning course. I averaged about 38 XP per day for 33 days, according to my activity log; there were 9 days when I did less than 30 XP, and 3 days when I did less than 10 XP, including 1 day when I did no Math Academy work at all. I took 8 quizzes and did re-takes for 3 of them; for 1 quiz I got 0 XP the first time.

All in all, I spent just over a month taking Linear Algebra, a relatively short time that’s due to already learning a fair amount of linear algebra in the Mathematics for Machine Learning course. (When I started the Linear Algebra course, Math Academy indicated that I had already completed 75% of the course topics.)

I know eigenvectors

As I’ve mentioned multiple times, my original interest in Math Academy was sparked by my desire to finish learning linear algebra, a process that I abandoned short of learning what an eigenvector was. I am happy to relate that I now know what an eigenvector is, as well as an eigenvalue.

This is not knowledge in the “knowing what” sense; that I could have gotten from Wikipedia or (if I wanted to be au courant) by asking an LLM. It’s “knowing how” knowledge, the result of filling lots of sheets of paper calculating eigenvalues and their associated eigenvectors, and then using them in various contexts (e.g., singular value decomposition and principal component analysis). Do I know all of this cold, able to employ any of those techniques with full confidence and automaticity? Not quite yet, I’ll no doubt need further review sessions. But I can’t look at a matrix anymore without thinking about computing its eigenvalues and eigenvectors.

More Math Academy musings

It’s also a tradition for me to include in these updates more of my thoughts on the Math Academy system itself. Here are two for this post:

Math Academy and Alpha School. In my post on Joe Liemandt’s intterview about Alpha School, I discussed TimeBack, the software stack (“education OS”) used by Alpha School and being licensed to other private schools. As it happens, TimeBack includes a number of third-party learning systems, including Math Academy. In fact, according to a TimeBack developer, Math Academy is among the most-used applications in the Alpha School TimeBack platform. A subsequent tweet details the criteria used for selecting applications for inclusion; it’s clear that Math Academy does well on all of them.

It’s interesting to speculate whether Alpha School is paying for individual Math Academy subscriptions for each student, or whether Alpha School is using Math Academy under some sort of enterprise site license. Based on various things I’ve read, it appears that the TimeBack system wants and needs clear feedback on student interaction with the learning application, and that’s difficult to do with a third-party application. Joe Liemandt also has a goal of deploying a version of TimeBack to a billion children, and that will be difficult with Math Academy’s current pricing. Thus I wouldn’t be surprised to see Math Academy and other third-party applications eventually replaced in the TimeBack stack with similar applications developed in house.

Does Math Academy need a “slow mode”? In a recent post titled “Frustrated and Confused,” Reddit user cmredd sounds off about Math Academy: “I think I’m quite close to cancelling my subscription unless a few (basic but key) things are fixed or added.” Their core complaint is that Math Academy doesn’t provide enough reviews, especially for students who only have enough time to do a few XP per day (e.g., 15 or less), and doesn’t provide a simple way for students to request more reviews. Hence the idea of a “slow mode” for such students that prioritizes reviews of prior material over the introduction of new material.

I made a number of comments in response to that post, and won’t repeat them here. But I did want to add that The Math Academy Way has an extensive set of FAQ chapters that respond to various questions and concerns about how Math Academy works, including some touching on the points that cmredd raises. These FAQs are well worth reading at length. Whether you agree with Math Academy’s general approach or not, there’s no question that its developers can provide coherent and onsistent answers for why it works the way it does.

However, I think there is a question about how useful Math Academy can be for a student without a good mathematical background who can spend only a few minutes per day on the system on average. I’m not a believer in XP-maxing (like people who brag about doing 100+ XP per day), but it seems like it wouldn’t be cost effective to pay $49 a month for a service where you can do (say) only 10 XP or less a day on average (e.g., just a single lesson, review, or quiz), need to do lots of reviews due to experiencing difficulty with the material, and therefore may take a year or even two to finish a given course. As I’ve written before, Math Academy is not for everyone, and I think such students would be better served by looking at alternatives.

(The folks at Math Academy agree, and it factors into their economic model. From the response to the FAQ “Why isn’t Math Academy free?” in The Math Academy Way: “Math Academy . . . must be priced in a way that the company’s solvency is not dependent on a massive user base.”)

Where to next?

As discussed above, I’ve now completed my original goal of learning linear algebra (at least to a first level), and in only 9 months compared to my original estimate of 1 year. The question now is, what to do next?

In my last update, I floated the idea of taking Multivariable Calculus and then Probability and Statistics, rounding out my knowledge of topics that the Mathematics for Machine Learning course provided an introduction to.

However, now that I’ve finished Linear Algebra ahead of schedule I’m thinking of instead taking Calculus I and then Calculus II, Math Academy’s university-level introductory calculus courses. There were areas of calculus that I didn’t get lessons on in Mathematical Foundations I and II, like hyperbolic functions, and I’m still a bit shaky on topics like differentiation and integration of the less common trigonometric functions. I’d like to have more automaticity on those topics before tackling the full Multivariable Calculus course.

But no matter what I decide, I plan to be back here in a few months writing about the successful completion of another step in my Math Academy journey.

My usual disclaimer applies: Not everyone will share my opinions on Math Academy, and not everyone will want to use the Math Academy system as I do. But my comments may be useful or interesting to at least some people.

Steady as she goes

Im my last update I opined that keeping to a reasonable XP goal each and every day was the best approach to maintaining steady progress in Math Academy courses. For Mathematics for Machine Learning I set a daily goal of 40 XP. According to my activity log for the 105 days I spent in the course, I averaged 48 XP per day. There were only 11 days when I did less than 40 XP, and only one day when I skipped Math Academy entirely and didn’t do any XP at all.¹

At the other end of the scale, there were only five days where I did 70 XP or more, and only two days when I did 100 XP or more. (One of these was the last day of the course, when I was close enough to completing it that I decided to just go the extra mile and get it done; that was also my maximum XP total at 102.) In general my strategy has been to do enough work each day to get at least 40 XP. The major exceptions to this were when I failed quizzes. Then I would do reviews and retake the quiz the same day, which padded my XP total a bit.

Coach is always right (or you should find a new coach)

One of the things people bring up from time to time is wanting a lot more flexibility in deciding which order to do lessons. This is also related to the complaint made by Oz Nova in his article “A balanced review of Math Academy” about the rigidity of the knowledge graph:

Why do some learning resources designed for autodidacts—such as Math Academy . . .—rely so heavily on dependency graphs? The generous answer may be that it’s unrealistic for the learner to know an appropriate ordering, and perhaps motivating for them to be shown the “ideal.” The cynical answer is that these programs take the idea of mastery learning too far, and have become dogmatic. . . . After all, it’d be easy enough to present users with a suggested sequencing without strictly requiring that it be followed.

A minor nitpick: Math Academy does not completely impose a strict ordering on students. Yes, the system decides what topics to present next, but when a student is presented with a list of lessons and reviews, they do not have to do them in the presented order.

But, as it happens, I am fairly religious about following the sequence of topics suggested by Math Academy: I do any suggested reviews first, going in sequence from first to last presented, and then do all the suggested lessons in sequence from first to last presented. I also generally do any presented quizzes as soon as they are presented, and only rarely choose to postpone them.

If we take at face value Justin Skycak’s analogizing learning math to developing athletic talent, then in essence I’m just following the instructions of my “coach.” Presumably the people who developed Math Academy know a lot more about math education than I do, and they have the benefit of lots of data about what works well and less well with Math Academy students. I‘m therefore not inclined to question their judgment about course and topic sequencing, any more than a freshman player would presume to tell their coach that they should change the content and sequence of their drills.²

Of course, sometimes players chafe at a coach’s instructions, and find that the coach’s approach doesn’t match well with their own talents and style. It‘s also true that some coaches are more competent than others. If a player finds they’re in a less than optimal situation, they can always find another coach, transfer to another school, or ask to be traded. As I’ve mentioned before, Math Academy is not for everyone.

“Among the chief glories of Western civilization”

Oz Nova also has some things to say on the topic of self-motivation and why (in his opinion) schemes like XP goals, leaderboards and leagues, and related Math Academy features fall short:

Every person has their own mix of motivation and the last thing I want to do is to judge another’s. But speaking personally, if a book promises to help me grapple mathematically with the chief glories of Western civilization, now THAT might sustain my interest.

This is from the introduction to Differential Equations with Applications and Historical Notes by George F. Simmons, a book as close as possible to perfect, for me, for this topic. It is not just motivating but stirring, driving, elevating. By math textbook standards, it overflows with fascination, excitement, even love.

The Simmons treatment is so vibrant that something like Math Academy sits lifeless in comparison.

Since I’ve had trouble learning about differential equations and would like to know more about them, I got a copy of Simmons’s book. I found it to be well-written and sprinkled through with information about the historical development of the topics covered, including mini-biographies of the mathematicians who developed them.

Is this motivating? It’s hard to say. As I’ve previously mentioned, I have a practical approach to learning mathematics, primarily focused on what I can do with it, so I’m probably not the best person to judge. I also often find myself distracted by sidebar material like this, so much so that I neglect the main discussion.³ From that point of view there’s a lot to be said for Math Academy’s “lifeless” approach.

Having said that, it’s possible that Math Academy might benefit by including more material on the historical background of the topics in its courses. Perhaps this could be included as a reward (not a sidebar), for example, after successfully completing a quiz or course.

Onward to (more) eigenvectors

As you may recall, my original motivation for signing up with Math Academy was to learn what an eigenvector was, and by extension to complete my study of linear algebra. The Mathematics for Machine Learning course covers various topics in linear algebra, multivariable calculus, and probability and statistics, and includes a fair amount of material about eigenvalues and eigenvectors, including their application to principal component analysis.

So in a sense I’ve already achieved my goal, as I now know what an eigenvector is. But I don’t want to stop here. My next task is to finish the real Linear Algebra course; I’ve already covered three quarters of its topics, per my dashboard page, so even allowing for having to relearn some topics I should be able to complete it well before the end of the year (my original timeframe). Then I’ll go on to Multivariable Calculus and Probability and Statistics, rounding out my knowledge of topics that the Mathematics for Machine Learning course provided an introduction to. That will be more than enough to occupy me in the coming months, then I’ll see where my Math Academy journey takes me after that.

The first (and at the time of writing, only) course offered on PhysicsGraph is Introduction to Quantum Computing. I decided to take advantage of a promotional offer (see below) and recently finished the entire course. And as I did with Math Academy, I decided to write a review.

PhysicsGraph compared to Math Academy

As noted above (and freely confirmed by Chris Sutherland, one of its two founders), PhysicsGraph is very much modeled on Math Academy. In particular, it shares the following features:

A finely-scaffolded knowledge graph. Physics, like mathematics, is a hierarchical field: you must know elementary topics by heart in order to understand more advanced ones. Both PhysicsGraph and Math Academy embody these dependencies in a “knowledge graph” for each course, a directed acyclic graph of individual topics (e.g., “quantum operations and quantum circuits”) showing which topics are prerequisites for other topics (e.g., “multi-qubit operations and quantum circuits”). As a student progresses through the knowledge graph, fulfillment of prerequisites (i.e., completing the corresponding lessons) unlocks lessons for more advanced topics.

To reduce cognitive load on students, ideally the topics in the knowledge graph should be “bite-sized,” each containing the minimum of material needed to advance to the next topics in the graph. This help ensure that students can fully master a given topic before going on to the next. (Justin Skycak of Math Academy refers to the ideal knowledge graph as being “finely scaffolded.”) For the most part this seems to be the case with PhysicsGraph, although there were some longer lessons (like quantum teleportation and superdense coding) where some might have preferred their being split in two.

As does Math Academy, PhysicsGraph publishes the knowledge graphs for topics in its course(s), so you can see the scaffolding for yourself. For example, see the knowledge graph for the Introduction to Quantum Computing course.

Experience Points (XP). Like Math Academy, PhysicsGraph assigns each lesson an XP value that presumably reflects the amount and difficulty of the material in that lesson. Students then earn XP for themselves by successfully answering questions, with XP bonuses handed out for successfully answering all questions in a lesson or review.

I found accumulating XP on PhysicsGraph to be easier than on Math Academy: I was able to do over a thousand XP in a week and well over a hundred XP per day. (I did 354 XP on one particularly productive day.) In contrast, I keep myself busy trying to maintain a pace of 40 XP per day on Math Academy. Part of the difference may be that I was already familiar with many topics in the Introduction to Quantum Computing course (e.g., matrix multiplication and vector spaces) from learning and practicing them on Math Academy.

Adaptive diagnostics. Like Math Academy, PhysicsGraph has students take a diagnostic exam before starting a course. The exam is used to judge the level of the student’s knowledge and which prerequisite topics they can safely skip. That frees the student from getting bored covering material that they already know.

I found there were only a couple of times where I felt PhysicsGraph assumed knowledge I wasn’t fully solid on, one being expressing complex numbers in polar form. (I had some problems with this in Math Academy too.)

Spaced repetition review. Like Math Academy, PhysicsGraph prompts students to periodically review previously-covered (and hopefully mastered) material, scheduled in such a way that a student reviews the material for a given topic at the point at which they’re in danger of forgetting it.

Math Academy has an elaborate algorithm for doing this (see my discussion of it), one that incorporates “implicit review” of material, i.e., reducing the overall number of reviews based on the fact that successfully reviewing a topic also implicitly reviews prerequisites for that topic. The PhysicsGraph website doesn’t contain detailed information about its SRS technology, but I wouldn’t be surprised to learn it has a similar scheme.

Im Math Academy the reviews tend to predominate as you reach the end of a course, so that you spend as much or more time reviewing previous material as you do learning the last few topics in the course. That wasn’t my experience with PhysicsGraph, which may be a good thing or bad thing: It’s possible that PhysicsGraph is not scheduling enough reviews to promote long-term retention.

Gamification. Like Math Academy, PhysicsGraph allows students to compare their progress to others by looking at a leaderboard displaying XP totals for the highest-ranking students. However, unlike Math Academy, participating in the leaderboard is not optional, and there is only a single leaderboard instead of the multiple “leagues” of Math Academy. I think participation in gamification schemes should be optional, since some students don’t want to compare themselves to others and don’t need the motivation provided by such comparisons. As for having multiple leagues, I don’t think that makes sense until the number of active PhysicsGraph students is one or two orders of magnitude larger than it is now.

Community forum. Like Math Academy, PhysicsGraph has a Discord server in which students can (at least in theory) share experiences, help each other with problems, provide feedback to the service’s founders, and so on. At present the PhysicsGraph server is relatively inactive, presumably because the number of students is still relatively small. (Even with Math Academy, which presumably has hundreds if not thousands of active students, only a relatively few people post to the Discord server; this is typical of social media in general, and I wouldn’t expect PhysicsGraph to be any different.)

Target market. PhysicsGraph will likely appeal to the same types of students that Math Academy does, namely homeschoolers, advanced K-12 students, university students, and adults of any age learning new fields for personal enjoyment (like me) or professional enrichment. In particular, based on discussions on the Math Academy discord server and elsewhere, taking a Math Academy course appears to be a popular way to prepare for a formal instructor-led course (e.g., at a college or university) or to supplement such a course while taking it. I expect PhysicsGraph to have a similar appeal, and in fact their proposed next courses (“Physics I” and “Physics II”) appear to directly address that use case.

No “freemium” option. Like Math Academy, PhysicsGraph is a paid service with no free offering (although it does offer a free trial with a money-back guarantee). In my opinion this is exactly the way it should be: These services offer real value to students, and students should be willing to pay for that value.

PhysicsGraph’s subscription pricing is currently $29 US per month, compared to $49 US per month for Math Academy. I think this is a fair price given the paucity of courses at present compared to what Math Academy offers; however I think there’s room for raising the price in future if PhysicsGraph can built out a complete set of undergraduate physics courses and even extend into adjacent areas like chemistry. (I should also add that PhysicsGraph’s current annual price of $199 US is an absolute bargain, representing a 43% discount off paying per month; with Math Academy, paying by the year gets you only a 15% discount.)

The lack of a free offering also means that PhysicsGraph, like Math Academy, has an easier path to becoming a sustainable business by bringing in revenue from day one. The long-term financial health of a service like PhysicsGraph is presumably important for its founders and employees. But it’s also important for those of its customers who are lifelong learners, who want to take more courses over time and continue to get reviews for those previously taken. And unlike a typical VC-funded “freemium” educational offering, a service that can sustain itself solely on revenue from students will also be resistant to adding intrusive ads, employing deceptive marketing techniques, and generally implementing other forms of “enshittification.”

There are also areas where PhysicsGraph differs from Math Academy, for better or worse:

No timed quizzes. Math Academy periodically tests students’ mastery of previously-covered material using timed quizzes, requiring the student to answer a series of questions within a 15-minute period. The results of the quizzes are then used to schedule additional reviews and to modify the difficulty of subsequent quizzes. Despite finding these quizzes to be stressful, I think they’re useful in exposing gaps in learning and determining how best to correct them. I hope PhysicsGraph sees fit to offer a similar feature in future.

No “free response” questions. Math Academy lessons, reviews, and quizzes typically use multiple-choice questions with five possible answers. However some questions require responses where the student types in the answer. (Math Academy provides some UI widgets to make this easier when entering formulas.) As with quizzes, I find these types of questions more stressful to do, because I can’t just test my tentative answer by comparing it with the presented list of choices; instead I have to double- or even triple-check my work. But, also as with quizzes, I think free-response questions help in learning the material. This is another feature that PhysicsGraph might consider for future implementation in cases where it makes sense.

No question answering when revisiting lessons. This is a subtle point, but one that I think is important: In both Math Academy and PhysicsGraph you can go back to previous lessons and go through the material again. In Math Academy you can also answer the questions again in the same manner as when taking the lesson originally, except that you do not receive any further XP. However, this is not possible in PhysicsGraph; when revisiting a lesson you see all the questions presented along with your previous answers (correct or incorrect) and the accompanying explanations. I think this is a mistake: a major reason to revisit a lesson is to solidify your understanding of that lesson’s topics, and working through the exercises again helps contribute to that. I think PhysicsGraph should consider implementing Math Academy’s approach.

No “AI-powered” marketing. Math Academy advertises itself as “AI-Powered,” and its website has a “How our AI works” web page. As I discussed in my review, I am not a fan of this strategy, which presumably seeks to exploit the current hype around ChatGPT and other LLM-based services to help market Math Academy. PhysicsGraph is blessedly free of this conflation of LLMs with the machine learning-based analytics underlying Math Academy and (presumably) PhysicsGraph. It’s not trying to advertise itself as A Young Lady’s Illustrated Feynman Lectures, and thank goodness for that.

(To be clear, I think LLMs may have a place in a service like PhysicsGraph or Math Academy. However I think they’re an unneeded distraction from the core functioning of the service, and figuring out when and where their use does make sense will require some thought and experimentation.)

Per-class pricing. For both Math Academy and PhysicsGraph the main offering is via a monthly or yearly subscription; I think this is the right choice for both services, since it provides ongoing revenue with which to improve the service and add more courses. However, as I previously noted with Math Academy, I think it’s useful to also offer students the option to pay a one-time fee for an individual course, including the ability to review material for that course after completing it.

PhysicsGraph has done this with the Introduction to Quantum Computing course, offering lifetime access for $99 US, discounted from $199 US as an early-bird promotion. The promotional price thus equates to over three months of the subscription pricing, with the regular pricing (when it goes into effect) amounting to over six months of a subscription.

I could have opted for a $29 per month subscription, knocked out the Introduction to Quantum Computing course in less than a month, and then cancelled the service. However, I went for the per-class price because I wanted to be able to review topics in the course on an ongoing basis without having to maintain a subscription.

PhysicsGraph compared to Quantum Country

PhysicsGraph is not the first attempt to teach the basics of quantum computing and leverage spaced repetition to help the reader remember them. A couple of years ago Andy Matuschak and Michael Nielsen created the Quantum Country website, billed as “a free introduction to quantum computing and quantum mechanics . . . presented in a new mnemonic medium which makes it almost effortless to remember what you read.” It covers much of the same material as the PhysicsGraph Introduction to Quantum Computing course, including a discussion of quantum teleportation.

How does Quantum Country compare to the PhysicsGraph quantum computing course? My snap judgment is that Quantum Country is superior as a scientific essay for a knowledgeable audience, and makes an honest attempt to sweat the details (as opposed to hand-waving over them). However, I think it’s inferior as a way to actually learn quantum computing basics.

First, and most trivially, Quantum Country assumes a basic level of knowledge that some readers won’t have, to be “comfortable with complex numbers and with linear algebra—vectors, matrices, and so on . . . [and] with the logic gates used in conventional computers—gates such as AND, OR, NOT, and so on.” That’s not a problem with a scientific essay, and as the authors note there are other resources to learn those details. But it is a problem for readers coming into the topic relatively cold.

Quantum Country also spends some time on the history of the ideas behind quantum computing and the context for its development, and includes a fair amount of material on the principles and benefits of spaced repetition. If you subscribe to Justin Skycak’s theory of skill development then a lot of that is arguably superfluous—like teaching someone how to play basketball by starting out with a discussion of the high-level principles that make it a great game.² PhysicsGraph’s (and Math Academy’s) approach is more like “let’s have you first learn how to dribble.”

Next, Quantum Country material is not divided up and scaffolded as it would be in PhysicsGraph or Math Academy; it’s more like an essay where the reader is continually led along. This may be a small point, but I think it’s important for a student to have a small sense of accomplishment after finishing one lesson and before beginning the rest.

I think it’s also important for the system to gate access to further lessons on successfully completing the current one; with Quantum Country you can completely skip (or wrongly answer) all the spaced repetition questions and just go on to read further without necessarily understanding what you’ve read already.

Finally, in various places Quantum Country mimics the “this exercise is left to the reader” practice typical of traditional textbooks. This is contrary to the instructional philosophy behind both Math Academy and PhysicsGraph, in which all topics should be fully explained (even over-explained), with the role of the student then being to practice on their own applying the explained techniques to problems that are very similar (sometimes only trivially different) to those covered in the instructional material.

However, Quantum Country does have one feature that I think might be profitably considered for PhysicsGraph (and Math Academy): it sends email messages to remind readers when reviews are due. Math Academy assumes a model in which students are accessing the system every day, will see offered reviews in the normal course of events, and do not need additional reminders.

PhysicsGraph also assumes that students are motivated to check the system on a regular basis, in part because they’re paying for a continuing subscription. But what about students (like me) who pay for lifetime access to a single PhysicsGraph course? There may come a time when reviews for that course are due only every few days, or even every few weeks or months. In that case it would be useful to get email reminders when reviews are due.

Learning to teleport quantum states

I think there are multiple areas where PhysicsGraph could be improved. In particular, after I completed the course and started to get reviews for various topics, I had real difficulty completing the reviews for the last topics in the course, superdense coding and quantum teleportation. Above and beyond any intellectual deficiencies on my part, I think this may be due to at least two factors:

First, and most important, I think not having periodic timed quizzes is a real lack. There have been multiple occasions in Math Academy where I didn’t do well on quizzes and had to go back and do additional reviews and even revisit lessons. I think this really helped me better understand and retain the material. This is missing in PhysicsGraph, and as a result I wasn’t doing as much review as I should have been.

I also think some of the topics in the Introduction to Quantum Computing course could be even more finely scaffolded than they are. As one example, I think the lesson on separable and entangled states and Bell states might be better split into two. I’ve had major problems remembering the symbols for the Bell states and their associated quantum states; that might have been at least partially remedied by having a separate lesson (along with associated questions and reviews) dedicated to learning about those states.

But even in its current somewhat embryonic state, PhysicsGraph is a fun and effective way to learn physics. The choice of quantum computing for the first course is an excellent one: The material is understandable with only a minimum of mathematics knowledge, and the course itself teaches any additional mathematics needed.

The topic is also intrinsically interesting. Quantum mechanics, on which quantum computing depends, is a shining example of our ability to understand the world and manipulate it, the theory enabling everything from your cat’s laser pointer to the laptop on which I’m writing this article. Quantum entanglement is a phenomenon that Einstein found baffling (he called it “spukhafte Fernwirkung,” “spooky action at a distance”) and the ability to teleport quantum states is one of its most surprising consequences.

Yet at the end of the Introduction to Quantum Computing course, after all the preliminary lessons about Kronecker products and Bell states, PhysicsGraph will take you to a point where you can not only understand how quantum teleportation works, but why it must work exactly that way. Doing it in practice is far from trivial—real magic is hard—but you will have unlocked the spell that underlies the trick.

I myself am not planning to subscribe to PhysicsGraph on an ongoing basis. My personal interest is in learning more about data science, and to that end I’m working through the relevant Math Academy courses to learn the needed background material. My interest in physics these days is a relatively casual one that can easily be satisfied through other means.

However, if you are interested in really learning physics, and not just reading popular articles or watching videos about it, I strongly recommend that you check out PhysicsGraph. It’s a worthy attempt to apply to learning physics the proven educational techniques—direct instruction, spaced repetition, and so on— implemented in the Math Academy service. PhysicsGraph is still under construction, but I have confidence that the people behind it will end up building something great.

Earlier this year I published a series of posts about Math Academy, summarizing the book The Math Academy Way and reviewing the service itself based on my experience thus far in completing the Mathematical Foundations II course. I’ve now completed the Mathematical Foundations III course as well and am starting the Linear Algebra course, the key to my quest to know what an eigenvector is Mathematics for Machine Learning course.

This is a good time to stop and present my updated thoughts on the Math Academy experience. Not everyone will share these opinions, and not everyone will want to use the Math Academy system as I do. But my comments may be useful to at least some people.

Class completions are the only milestones worth celebrating

Lots of people post daily or weekly about how many XP they’ve done that day or week, or celebrate their passing a certain milestone like 4,000 or 5,000 XP. I have nothing against such people, but I just don’t see the point of that. An XP total is an arbitrary figure and doesn’t necessarily bear a direct relation to what you’ve learned.

If your goal is to drive from New York City to Los Angeles, or from Paris to Berlin, it doesn’t matter how many miles or kilometers your car’s odometer shows, or even how many miles or kilometers you drive in a given day. All that matters is whether you reach your destination. You can celebrate when you get there. (As a corollary, that means I won’t be posting again about my Math Academy progress until/unless I complete the Linear Algebra Mathematics for Machine Learning course.)

But if you do have a burning desire to see my progress and verify it, you can look at my activity log for the time I spent doing Mathematical Foundations III and my certificate of completion for the course.

League participation is optional and potentially distracting

As I wrote in my earlier posts, I started out participating in the league updates. It was fun to a certain extent to compete against other people, but eventually I got a little too stressed out not just trying to qualify for promotion, but trying to be in the top 1 or 2 slots for the week.

I can see where competing against others and vying for promotion might serve as an external source of motivation for many people, and there’s nothing wrong with that—it’s good that Math Academy includes that feature. However, in my case I was motivated enough already, and doing leagues as well was a bit too much of a muchness. So, I eventually decided to stop worrying about the leagues and just concentrate on making progress in my course. As of now I see no reason to regret that decision.

You should try to do something every day (but not overdo it)

Even though I quit the leagues, I still made it a point to do lessons every day, ideally enough to exceed the 50 XP per day quota I had set for myself. I think I succeeded in that goal—I can’t recall any days on which I didn’t do anything at all, although there have been a few days when I just did one or two reviews.

Many areas of Math Academy appear to assume a student will be doing lessons five days a week, but I strongly recommend scheduling yourself to study every day if you can do it. There will always be emergencies that take precedence over doing a lesson, but I think it’s too easy to slack off and fall out of the habit if you deliberately miss a day or two.

At the same time, I encourage you not to overdo things. This is one reason I don’t pay much attention to people posting insanely high daily or weekly XP totals. Maybe they have the spare time to spend several hours a day working Math Academy exercises, but you probably don’t. Consistency, not speed, is the key; slow and steady wins the race.

During the Mathematical Foundations III course I typically found myself exceeding my 50 XP per day quota—it was too tempting to do one more lesson, or to do a review as soon as possible after missing a quiz question. For the Linear Algebra Mathematics for Machine Learning course I’ve decided to lower my quota goal to 40 XP; I have another projects I want to work on, and want to preserve time for them. Even at 40 XP a day I should be able to achieve my goal of finishing Linear Algebra Mathematics for Machine Leaning by the end of the year.

Math Academy should let you select your desired course sequence

When I finished Mathematical Foundations III, the system immediately began offering me lessons for the Mathematics for Machine Learning course. This is apparently because this is the default track that the system lays out for you. But, as I’ve noted multiple times, I don’t want to take Mathematics for Machine Learning, at least not now. I want to take Linear Algebra. (Yes, I know that Mathematics for Machine Learning includes a large chunk of linear algebra, but I want the full course of it.) (UPDATE: As noted above, I decided to take Mathematics for Machine Learning after all, and come back to the full Linear Algebra course later. But I still stand by my comment in the next paragraph.)

Unfortunately, I can’t find any way in the Math Academy interface to change the track that I’m on. The system knows (or should know) what courses are prerequisites for other courses, so it would be nice if I could see the tree of possible courses, select a goal to work toward (which for me right now would be completing the Probability and Statistics course), and then offer the courses in sequence needed to meet that goal.

Math Academy is (still) a great service

Despite the nitpicks I have about certain aspects of Math Academy and the online interface, I still think Math Academy is a wonderful service and have no regrets whatsoever about signing up for an entire year. I can’t speak to how it would be as a service for K-12 students, but if you’re a self-motivated adult learner interested in picking up math for the first time or as a refresher, Math Academy is well worth the money.

As for me, I have Linear Algebra Mathematics for Machine Learning to look forward to, and more courses currently being offered that I’d like to take beyond that. And with Math Academy adding more new courses this year, I’ll probably find a few more that I’d be interested in. Math Academy offers enough to keep me busy through the end of my current subscription, and I suspect I’ll be renewing it for next year as well.

My current status

As of the time of writing, I’ve completed the Mathematical Foundations II course and am about 28% of the way through the Mathematical Foundations III course, having accumulated a total of over 3,600 XP thus far. I’ve settled down into a routine of doing at least 50 XP a day, sometimes a bit more than that. I have never gone a day without doing at least something.

If I maintain my current rate of progress, I hope to complete the Mathematical Foundations III course sometime in late May or June (but see my complaints in the prior post about progress estimates!), and will then start the Linear Algebra course. As I mentioned previously, my goal is to finish the Linear Algebra course before the end of 2025, and then to complete the Multivariable Calculus and Probability and Statistics courses by the end of 2026.

Is Math Academy worth the money?

Let’s start with the obvious topic, money. If I meet the goals discussed above by the ends of 2025 and 2026, I will end up spending over a thousand dollars on Math Academy. That’s a lot of money, even for someone like me with a well-paying job. Then I’ll be faced with the choice of paying for the service a while longer, to make sure I get any needed reviews and won’t forget what I learned. (See my suggestion in the last post for a reduced-price offering that just covers reviews.)

However, on balance I think it was and will be worth it. Learning mathematics is both a hobby for me and supports other hobbies I have, and I’ve spent similar amounts on hobbies in the past. And it’s certainly true that spending several years on a less costly approach (reading a textbook and doing its exercises) didn’t work out for me. So, I’m happy to pay for a service that looks like it will.

Do you actually learn anything?

I think I’m learning some mathematics with Math Academy, but am I really? Well, I now have a much better handle on some topics that I previously had never encountered or had trouble remembering. The most notable of these are the synthetic division method for polynomials, the various trigonometric identities, and differentiation of products and quotients of functions.

I still struggle remembering some things, like the derivatives of the secant and cosecant functions, but I hope that will get better in time. In the meantime I sometimes resort to deriving those from the derivatives for sine and cosine. (This works fine in lessons and reviews but kills performance on time-limited quizzes.)

One thing I will say is that the “scaffolding” aspect of the Math Academy system means that often what I’ve learned is the simplest possible case, and I would be at a loss dealing with more general cases. For example, the Mathematical Foundations II course teaches using synthetic division to divide a simple linear polynomial factor (e.g., x-3) into a polynomial of higher degree, but not how to divide an arbitrary polynomial into another.

I don’t see this as a drawback, though. It’s necessary to master simple cases before taking on more complex problems, and the confidence gained from knowing how to solve simple cases gives one a step up when it comes to learning about harder ones.

Does speed kill learning?

One criticism levied at Math Academy is that it overly emphasizes speed of learning. For example, this criticism shows up in Michael Pershan’s critique of the Math Academy system and in student quotes from an article about the original Math Academy program that he links to (“We do too much in not enough time”).

It’s certainly true that the developers of Math Academy put great weight on the speed of learning (“accelerates the learning process at 4X the speed of a traditional math class” is a key selling point). And at $49 a month there’s a built-in incentive to go as fast as possible, to get the most bang for the buck and avoid spending too much by stretching out your course time.

This emphasis on speed also shows up in the leagues and leaderboard system, which encourages students to compete on how fast they can pile up XP. Finally, it’s a common occurrence on X and similar forums to see people posting their XP totals in a way that’s reminiscent of people posting their high scores in a video game. It prompts the question: “OK, you did 7835 XP, but what mathematics did you actually learn?”

I’m sure that the Math Academy developers will respond that learning rapidly is perfectly compatible with learning well. Certainly The Math Academy Way doesn’t mention any drawbacks to trying to speed run through courses; at one point it raises the possibility of getting through a typical course in 5-6 weeks (at 100 XP a day), and treats that as an unambiguously positive thing to do. (An included table goes even further and shows that by doing 160 XP a day you could complete a typical Math Academy course in three weeks.)

However, after having breezed through most of the Mathematical Foundations II course in less than a month, I’ve decided to deliberately slow down my pace. With the Mathematical Foundations III course getting into more topics I have less previous experience with, I’m trying to stick to my original goal of 50 XP per day, as opposed to trying to push up to and past the 100 XP a day mark. I also have other things I want to do in my spare time, and I don’t want to get to a point where I’m doing so many math exercises that I get sick of the whole process.

What about this “understanding” thing?

Another point of contention is whether Math Academy is really helping students to understand mathematics, or whether it’s just teaching superficial procedural skills. (Michael Pershan harps on this, for example, as does Dan Meyer.) Other comments have criticized the Math Academy system because it’s so different from the heavily proof-based way of learning mathematics embodied in various advanced textbooks, in which students build up their knowledge of particular areas of mathematics almost from first principles.

I’m not an expert in mathematics education, so I can’t really speak to the debates about whether Math Academy promotes understanding or not. I do read the explanations embedded in Math Academy lessons, though, and I do try to build up a basic understanding of what I’m reading.

As for proofs, the Math Academy developers are certainly aware of student demands for a more proof-oriented approach (see for example a post by Justin Skycak) and are apparently working to add more courses in that vein. That may or may not satisfy those who want Math Academy to be more like the textbooks they’re used to.

However, as a former applied math and physics major, my approach to mathematics is primarily instrumental: I want to know how to do math in support of other things that I want to do. I am less interested in learning mathematics for its own sake, and am very much not interested in an approach to math that is overly abstract and proof-focused. From my point of view the Math Academy approach is therefore very congenial.

Who else might benefit from Math Academy?

It’s been a while since I had to worry about school instruction in mathematics, whether for me or anyone else, so I can’t speak to the issues around using Math Academy for home-schooled students or mathematically talented students. But as an adult student of mathematics I do have some thoughts about Math Academy’s approach to that segment of the market.

If you base your impressions on what Math Academy students post on X, Reddit, and other online venues, you could get the impression that the typical adult Math Academy user is what the sociologist Tressie McMillan Cottom snidely called a “roaming autodidact”: “a self-motivated, able learner . . . simultaneously embedded in technocratic futures and disembedded from place, culture, history, and markets . . . almost always conceived as western, white, educated and male.”

Assuming that’s the case, one can understand some of the current product focus of Math Academy the company, including introducing more proof-oriented courses, as mentioned above, and especially adding courses like Mathematics for Machine Learning that cater to the latest hotness. They are doing what Clayton Christensen claimed incumbent vendors do, namely going up-market and catering to their most demanding customers.

But in so doing, is Math Academy closing off other potential market opportunities and leaving them to possibly be exploited by others instead? Let’s go back to Benjamin Bloom’s original research and recall that what got people’s attention was not that he was able to take a 95+ percentile mathematically-talented student and turn him [sic] into the next John von Neumann. Rather it was the claim that Bloom’s methods could enable an average student to reach much higher levels of achievement.

There are a lot of otherwise-intelligent adults out there who left off their mathematics education fairly early due to encountering difficulties and suffering ensuing math anxiety. What Math Academy could offer them is a firm grounding in actual mathematics, as opposed to a simplified exposition high on verbosity and very light on equations (the mathematical equivalent of the “physics for poets” courses some institutions have).

Let’s consider an alternative customer persona:

She [sic] is a former humanities major now working in a relatively non-technical job in a company or industry with a tech focus. She feels the lack of the mathematics training she never got, or dropped out of early due to encountering difficulties. She’s motivated to remedy that lack, and has even tried taking a couple of community college mathematics courses, but it just didn’t work out for some reason—maybe it was the teacher or curriculum, maybe she couldn’t make the schedule work due to work or family responsibilites.

She makes a reasonably good salary for a humanities major, but not nearly as much as the technical professionals she works with. She’s moderately progressive in her politics, and checks social media posts on Bluesky when she has a few minutes, because that’s where her IRL and online friends are.

How might Math Academy cater to her and people like her (assuming that it wants to)?

First, meet her where she is. Extend the word of mouth marketing efforts beyond X and Reddit to alternative platforms like Bluesky, which currently has over 30 million registered users. (PS: If anyone from Math Academy wants to set up a Bluesky account I’d be happy to reskeet their skeets.)

Second, further tone down the “AI-powered” messaging, or just ditch it entirely. As I mentioned previously, it’s a potential turn-off for many people, and she may be one of them. (And even if she’s OK with it, her friends may give her grief for using anything having to do with “AI.”)

Finally, consider some alternative product packaging. She doesn’t want to march her way through every course Math Academy has to offer. She’d likely be interested only in the Mathematical Foundations courses, and possibly only the first one or two courses in that series. Given the other demands on her time, she’s leary about committing to spend $49 month after month. She might instead like to pay a flat rate (amount TBD) for a single course like Mathematical Foundations I, and have a generous time limit for completing it, say six to nine months. (A 3,000 XP course could be completed within six months at a pace of 30 XP a day, or within nine months at a pace of 20 XP a day.)

Whether such an offering would find much take-up with this hypothesized “ex-humanities major” market is an open question. However, I think it would be a more attractive offering to that market segment than what the current service provides. And even if relatively few people take Math Academy up on the offer, I think there could still be important intangible benefits to reaching out to adult learners beyond the current core user base of tech-savvy folks. It’s not something Math Academy necessarily needs to do now—after all, the service is still technically in beta—but it’s something that might be worth considering in the mid-term.

We have LLMs now, why bother learning math?

Speaking of “AI” and the latest hotness, one possible response to Math Academy is, “Why bother?” I can ask an LLM to explain pretty much any mathematical topic to me (even in poetic form if I’m so inclined), and as LLMs improve their various capabilities I can just point one to a mathematical problem and ask it to solve it for me. What’s the point of paying $49 a month to Math Academy and doing all that work, when I could pay less (or even nothing) to get on-demand mathematical expertise with little or no work required?

There are responses one could make regarding why internalizing mathematics knowledge is important, especially if your job involves tasks that are mathematical or quasi-mathematical in nature. But that’s not my situation, so I’ll leave it to others to make that particular case.

My response is simpler: I enjoy doing math-related activities (like hobby data science projects), just like I enjoy writing, and I have no interest in outsourcing those activities to anyone or anything else. If I wanted to play the guitar, I wouldn’t be satisfied with paying someone to play the guitar for me. I also wouldn’t care that other people can play the guitar much better than I ever could. I would just want the satisfaction of mastering the guitar myself and being able to play a song. That’s how it is with me and mathematics, and no conceivable LLM could change that particular equation.

(As it happens, I did ask an LLM to explain to me what an eigenvector is. The answer was like a condensed and simplified Wikipedia article, and reading it was unsatisfying. I want to play the guitar myself.)

Whither Math Academy?

Finally, what of Math Academy itself? When I got a copy of my transcript after completing Mathematical Foundations II, it listed my student ID as being above 12,000. Assuming that IDs are assigned sequentially and not randomly, that means that over 12,000 people have used the Math Academy system and enrolled in a Math Academy course.

If all those people were still subscribed to the service then Math Academy would have monthly revenue of about $600K or about a $7M a year run rate. In reality the number of current subscribers is presumably somewhat less than that (given that many subscribers may have stopped after a few months), but on the other hand the subscriber base seems to be growing reasonably rapidly. So, my Fermi estimate is that for 2025 Math Academy could be up to a $10M-a-year business with up to 20 employees.

As I mentioned before, I can’t speak to the market fit for home schoolers or mathematically talented youth. However, I think Math Academy has achieved good product/market fit for that group of people of which I’m a member: adult learners with at least some mathematical background, a desire or need to learn or re-learn math, and a reasonable amount of spending power, typically due to working in IT or other high-tech fields.

Math Academy is clearly catering to that market by its choice of courses to add, including the Mathematics for Machine Learning course and proposed computer science courses, expanding from its core mathematics curriculum into adjacent areas. It can further broaden the market by some of the sales and marketing initiatives already discussed, including PPP pricing for non-US students, group and family discounts, and the like. These measures will lower revenue per user, but will expand the total addressable market.

I suspect that Math Academy could grow its subscriber base to 10x its current size, and its revenue almost as much, and could do so on a relatively accelerated schedule. Given that it’s (presumably) cash-flow positive today and should stay that way barring misdirected overspending, it looks to be a business that will be around for the long haul.

What about the larger claims made on Math Academy’s behalf? There I remain unconvinced. I don’t believe that Math Academy and the ideas about mathematics education it embodies will have any significant impact on traditional educational institutions in the near or medium terms. (The reactions of people like Michael Pershan and Dan Meyer are early indicators of that.) There are people and organizations working to change that mindset (and new ones popping up all the time, like the just-established Center for Educational Progress), but I suspect their’s will be the work of a generation and perhaps longer.

Despite Math Academy’s claims to be research-grounded and evidence-based, I’m also skeptical that we’ll see much if any research focused on Math Academy itself, trying to determine whether its pedagogy really has any advantages over the alternatives. The Math Academy folks are clearly too busy to do this, and I’m not sure anyone else will be willing to take on the task. So, all we’ll have is anecdata, anecdata that will be untrustworthy given the selection effects inherent in who uses the service. (I should add that my own anecdata shouldn’t be trusted any more than anyone else’s.)

But, in the meantime, I’m happy to have a service that seems to work for me, happy enough that I just upgraded my Math Academy subscription from monthly to annual. So, with that I’ll end this series and go back to working through Mathematical Foundations III. If you’ve checked out even one or two of my posts on Math Academy, thank you for reading!

Inital course selection

As I’ve mentioned many times previously, my goal is to learn what an eigenvector is, and for that I’ll need to take the Linear Algebra course. However I was not so deluded as to think I could just jump into the course, since it had been a while since I’d seriously studied math.

I therefore elected to start with one of the Mathematical Foundations courses recommended for adult learners. Unfortunately my delusion, while not severe enough for me to start the Linear Algebra course right away, was severe enough to make me think I just needed to take the Mathematical Foundations III course, the immediate prerequisite for the Linear Algebra course.

That was a major mistake, as discussed in the next section. I did a cursory check of the “Overview,” “Outcomes,” and “Contents” sections in the Mathematical Foundations III course description. However, in retrospect it would have been nice to have some additional information that I could have used to make an initial assessment of course appropriateness even before taking the diagnostic exam.

For example, suppose that the Mathematical Foundations III course description had had a minimal discussion of prerequisites, like “Among other things, this course assumes that you know how to factor polynomials using synthetic division, how to use trigometric identities like the law of sines, and how to differentiate the product or quotient of two functions.”

Then I would have immediately known that the Mathematical Foundations III course was too advanced for me, and I should take the diagnostic exam for the Mathematical Foundations II course instead. (Yes, I could have also just looked at the content for Mathematical Foundations II to see what that course taught, but I was being lazy and didn’t think to do that.)

The diagnostic exam

Instead I started taking the diagnostic exam for the Mathematical Foundations III course. It was a brutal and dispiriting experience, as I found myself being able to answer only a small fraction of the questions correctly. For most questions I didn’t even know how to go about finding an answer.

At this point I was internally shouting, “Stop! Let me cancel this exam and go back to square one!” However, I didn’t see any obvious way to stop the exam and switch to the diagnostic assessment for Mathematical Foundations II. So I soldiered on, finished the exam, and then was thrown into the beginning of Mathematical Foundations III.

As with the diagnostic exam, I just wanted to stop, go back, and try the Mathematical Foundations II course instead. Unfortunately, I couldn’t figure out a way to do that from the main course screen. It wasn’t until I got an email announcing the results of the diagnostic exam that I was enlightened: you can change courses by clicking on the profile icon in the upper right, selecting “settings,” and then selecting “Course” from the “Settings” page.

I then took the diagnostic exam for Mathematical Foundations II, did much better, and was soon embarked on my math learning journey.

Lessons and questions

Once I started the Mathematical Foundations II course, I found I really liked the way Math Academy organizes lessons and the accompanying questions. The lessons are relatively short, so I can do them whenever I feel like; I’ve typically been doing a couple in the morning before breakfast, a couple during lunch, and then more in the evening after dinner.

I originally planned to accumulate 50 XP a day, seven days a week. (I thought it important to do some work every day to not get out of the habit.) As it happened, I did over 90 XP every day for the first few weeks before cutting back (as discussed below).

I did more than anticipated for two reasons. First, I got a little competitive when looking at the league standings; more on this below. I also found doing the lessons and exercises to be a fun alternative to scrolling through social media. You can get a good sense of my progress by looking at the activity log generated by the Math Academy system; it covers the time from when I started until just after I completed the Mathematical Foundations II course, almost a month in total. (However, as noted below, I came to rethink my practice of overdoing things like this.)

My working method when doing lessons has been to read through the first example in each lesson, and then to try to work through the second example myself before reading the explanation for it. I found this to be a useful low-stress way to prepare for the first question following the examples. The questions themselves are relatively straightforward to answer if I understand the examples. I always try to work through the question first before looking at the multiple choice answers.

This generally works well: Either I get the answer right or I screw up something, choose the wrong answer, and get corrected. However, there have been occasions when I got an initial answer, didn’t see it listed among the five choices, and went back to try to understand what I did wrong. Some may consider this a bit of a cheat, but it’s inherent to a system organized around multiple choice answers.

(The questions for which you need to type in something avoid this problem, but as noted in my previous post they have their own problems when trying to access the system on a tablet. Also, in cases where there are multiple values in the answer—e.g., you need to find two values a and b, instead of a single value a—the Math Academy system tries to mitigate the problem of students just looking at the multiple choices and then trying to verify each one. It does this by requiring the student to enter or choose a sum or product of the answer values—e.g., a + b—rather than presenting the values themselves in the multiple choices.)

The questions are generally variations on a theme—sometimes very minor variations. Given that, I can see why some people would hate hate hate this way of instruction. There seems to be a macho attitude on the part of some students that learning mathematics is not supposed to be easy, that one has to conquer the exercises in textbooks like “Baby Rudin” and “Papa Rudin” before one can consider oneself mathematically knowledgeable. It’s like a hazing ritual, in which each generation has to suffer in order for the prior generation to consider their own suffering justified and worthwhile. The people who created the Math Academy clearly do not share this attitude, and I for one am very glad of it.

When I answered a question incorrectly, most of the time it was because I messed up something in working the problem: I misread the question, made a sign error, or screwed up when copying a term from step to step. The diagnostic exam has a “I made a silly mistake” option that allows you to retry a question, but the regular course questions do not. I think this is the correct approach: if you make silly mistakes on a regular basis then your goal should be to train yourself to be more careful in working a problem, including checking the result.

I don’t guess at answers, but there have been times when I accidentally got the correct answer: I didn’t quite know how to work the problem, but I stumbled on the right answer anyway. There’s no option to tell the system “I got this right, but I really didn’t know how to do it.” I originally thought it might be a good option to add, if for no other reason than to get a needed extra review, but this happens so infrequently that I doubt it’s worth it.

Reviews

I’ve previously tried out spaced repetition systems, most notably Anki, for learning various topics. I soon found myself overwhelmed by the sheer number of items I needed to review each day, a number so large that I eventually gave up in despair—it was sheer tedium to work through them all, especially when first learning the information.

Fortunately, Math Academy does not have that problem. On average I had to do less than three reviews of topics each day; a few days there were no reviews at all, and on a couple days there were many as six to eight. Each review topic had just a few questions—plus I’ve noted that the review ends early if you answer the first three questions correctly.

I consider that amount of review quite reasonable. The Math Academy Way claims that the Math Academy system leverages the hierarchical nature of mathematics to reduce the amount of needed review. In my experience, that claim is justified.

However, there are cases where I might in fact like to have more reviews, namely when it comes to learning sets of related facts. A good example is the set of standard angles on the unit circle and the values of trigonometric functions for those angles: the sine, cosine and tangent of 30 degrees (π/6), the sine, cosine, and tangent of 45 degrees (π/4), and so on around the circle. I found myself unable to recall all of those values instantly, and had to resort to calculating them in some cases. (For example, the sine, cosine, and tangent of 150 degrees can be easily derived from the values for 30 degrees.)

I’d rather not slow myself down by having to derive some of these on the fly. I’d rather be able to recall them instantly, just as I can instantly recall that 4 * 5 = 20. I could certainly put those facts into Anki or a similar spaced repetition system, but it might be nice to have the Math Academy system allow for special reviews of facts like that. (It appears that Math Academy may be planning something along this line, based on tweets from some of its employees.)

Quizzes

I had to do less than one quiz a day, of which some were retakes due to my not doing so well the first time. As with reviews, I consider this an acceptable number.

I found the quizzes themselves to be a bit stressful due to the time limit. On several occasions I had to leave one or two questions unanswered. On other occasions I rushed through problems and didn’t check them properly. As as result I scored lower than I would have liked several times, with the system reacting by having me take the same quiz again. Of the 15 unique quizzes I was presented during the Mathematical Foundations II course, I had to retake five.

Despite raising my stress level a bit, I thought the quizzes were useful and reasonable. I think the key to doing well on them is to develop “automaticity” in the sense used in The Math Academy Way, so that you can solve the presented problems without having to think too much about exactly how they should be solved.

Leagues and leaderboards

The leagues and associated leaderboards are an optional feature of the Math Academy system. I left them turned on initially because I wanted to see how they worked and how I compared with others. I was able to quickly advance through the lower leagues, often being the high scorer within my group. When this series started I was in the Platinum League (the middle league in the list of league ranked by exclusivity) and was about to be promoted into the Sapphire League. I found this aspect of the system to be fun and motivating, at least for a time—often I would do a lesson or two more just to improve my standings within my league.

Speaking of motivation: There isn’t much reward within the system itself for ranking high within your league, or for being promoted from one league to another. The period of competition simply ends and then you’re in a new league. It might be nice to have even a simple “Congratulations on being promoted!” message, or a “Hooray! You ranked first in your group!” Maybe the Math Academy folks think such things are unneeded or even undesirable, or maybe they just haven’t gotten around to implementing frills like this.

The flip side of the league and leaderboard gamification is that it encourages people to pile up XP simply for the sake of advancing in the leagues. You see this phenomenon on X as well, with people posting their cumulative XP. I don’t really need to be chasing the 100 XP a day mark in order to meet my own learning goals: 50 XP a day would have me completing the Mathematical Foundations III course by July, and then I’d have the rest of the year to finish the Linear Algebra course.

So, after I got promoted into the Sapphire League, I turned off the league feature. I’ve settled into a routine I can maintain, no longer have the distraction of invidious comparison with others competing for promotion, and have recovered some of my spare time for other things I enjoy.

Measuring course progress

Now that I’ve moved on to the Mathematical Foundations III course, one source of frustration to me is figuring out how I’m progressing within the course relative to my goal date for finishing it. (Note: The numbers I quote in this section are from when I first wrote it, a few days before publishing this post. I’ve progressed further since then.)

In the “XP Goals” section of the “Settings” section of the website, I’ve told the system that my goal is to do at least 50 XP each and every day of the week. The system then tells me on that same pqge that “At a pace of 350 XP per week, it’s estimated that Mathematical Foundations III will be completed by late May.”

However, if I go to the main learning screen, where the lessons are, it tells me that I am 24% through Mathematical Foundations III and that “Estimated completion is mid-July.” If I hover the cursor over that statement, it expands into a subwindow claiming that at a (boldfaced) pace of 49 XP I will indeed finish in mid-July. And if I hover over the ”24%” in a circle (next to where the page displays the course title), that subwindow also claims my pace is 49 XP a day. It further claims that I started the course on February 6 (correct), have a goal of ending the course on June 30 (also correct), and have done 691 XP in the course thus far (also correct as far as I know, from looking at my activity log).

Then it goes on to tell me that my “expected” progress should be 1260 XP, and that I’m therefore 569 XP behind schedule. It also tells me that the “total days” is 26 (with a parenthetical comment, “non-holiday weekdays”.)

This is all very confusing. Where does this figure of 49 XP come from? If I look at the activity log for the 8 days that I’ve been working on Mathematical Foundations III, I’ve been averaging 67 XP a day. And where does the “26 days” come from? There are a lot more than 26 days between now and June 30, about 50 all told if my calculations are correct, not even counting weekends and holidays.

(As an aside, The Math Academy Way assumes a 5-day week in its examples of learning speed and related topics. I’m wondering if that’s carried over into the Math Academy system, even in the case where a student—namely me—has indicated their intention to study every day.)

I’m just asking for a reasonable estimate of when I’ll be able to complete Mathematical Foundations III, and what pace I’ll need to maintain in order to make my goal date. Right now I’m not getting that, and I have limited trust in the numbers that the Math Academy web interface is showing me.

Course completion and results

I finished the Mathematical Foundations II course as my first month with Math Academy was nearing its end. As with being promoted to a higher league, this was a non-event as far as the user interface was concerned: I simply looked at the page one day and noticed that it was now showing my being in the Mathematical Foundations III course. (This was presumably because I had enrolled in Mathematical Foundations III originally, and so the course was in effect waiting for me to finish Mathematical Foundations II.)

The transition between courses was about as exciting as watching your car’s odometer roll over at the next 10,000 or 100,000 mark—especially when you get distracted and miss the exact time when it rolls over. I put many hours into working through all the course lessons, and it would be nice if the system would take note of that.

However, I should note that I did get a transcript and a certificate of completion for the Mathematical Foundations II course, as well as documents providing a course overview, a detailed list of course contents, and the accreditation letter for Math Academy itself. (Note to Math Academy folks: the course overview document has a typo: “Upon completeing . . .”)

These aren’t much use to me in my current situation (except to show skeptics that I really did use Math Academy before reviewing it), but they’re nice to have. I guess someone could use the certificates and transcripts to show a potential employer the level of mathematics they have learned (assuming the job requires using math as a core component). But, if I were an employer, could I really trust such evidence? After all, it’s possible that the student just breezed through the courses using an LLM to come up with answers.

Speaking of LLMs, that’s one of the topics I’ll be discussing in part 11 of this series, in which I offer my final thoughts on Math Academy.

Marketing

I have two comments here, one positive and one not so much. First, the positive:

Thus far Math Academy has (by its founders’ admission) spent no money whatsoever on advertising (Internet-based or otherwise), preferring to have its message spread by word of mouth. I think this is an excellent approach, especially for a service that is still in beta. I myself found out about Math Academy from a post on X, and since then I’ve seen multiple mentions of Math Academy on other services, including Reddit in particular (e.g., the /r/learnmath subreddit and—of course—the r/MathAcademy subreddit). Most of these mentions have been positive, from people who’ve used the service and like it. Your customers are your best salespeople, as the saying goes.

Math Academy staff have been active on these various platforms, providing in-depth responses to questions from people interested in signing up for the service. Again, this is an excellent approach, especially if the target audience is technically knowledgeable.

You can’t find Math Academy-related posts on all social media. In particular, I couldn’t find any Math Academy presence in the Fediverse (at least the section of it I have search access to) and the only post about Math Academy I’ve found on Bluesky was someone else’s repost from X. I wish Math Academy would establish a Bluesky or Mastodon account, even if it’s for nothing more than product announcements. There are lots of people like me who have moved their social media activity mostly or totally off of X.

Now for the not so positive: I am not a fan of Math Academy using the term “AI-powered” to describe its service. I think it’s a flawed strategy in at least three ways:

First, it’s a “me too” marketing strategy for a service that claims to not be a “me too” service. It seems to be trying to leverage the hype around large language models and generative AI to advertise a service in which the only “AI” appears to be traditional machine learning. I understand the motivation, but:

There are going to be lots of startups trying to ride the AI bandwagon with promises of personalized tutors that will talk to your child and patiently hold their hand while they learn. By leaning on the “AI-powered” slogan, Math Academy risks having itself be lumped in with a crowded field of companies trying to turn an LLM into A Young Lady’s Illustrated Primer. And if those companies fail (as many of them no doubt will), there’s a risk of having their failure taint Math Academy as well.

Second, hyping Math Academy as an “AI” application will also likely risk harming its image with that relatively large contingent of people for whom “AI” is synonymous with copyright violations, planet warming, and the immiseration of artists, writers, musicians, and many others in the name of VC-driven hypergrowth. “AI-powered” is a phrase that’s an instant turn-off for a lot of people who might otherwise be prepared to evaluate Math Academy on its merits.

Finally, “AI-powered” is “feature/benefit” marketing: “here is a product feature and here is the benefit it brings.” I strongly feel that Math Academy would be much better off with a “problem/solution” approach to marketing itself—or, if you want to get even more stark, a “pain/pain relief” approach.

(For example, for an adult learner like me the marketing message of Math Academy might be stated as follows: “You’ve forgotten more math than you remember, and you need or want to re-learn it. Math Academy helps you learn mathematics so that you never forget it again, and helps you do it in the least amount of time possible.”)

The main Math Academy website (the one you see if you’re not an active student) mostly gets this right: The headline is “Math Academy is the most efficient and effective way to learn math . . . guaranteed,” and there’s only four places on the page that reference “AI”: “AI-driven algorithms“ in relatively small print, a question “Is it really AI?” also in relatively small print, a larger headline “AI-Powered + Research-Driven Pedagogy” halfway down the page, and finally a small link “How our AI works” at the very bottom of the page.

The question “Is it really AI?” and the link “How our AI works” would cause the casual reader to expect that the answer would be “yes” and that clicking on the link would lead to a discussion of how Math Academy is leveraging LLMs—what most people would likely think of today if they heard the term “AI.” But instead the “How Our AI Works” page talks about Math Academy using an expert system, the knowledge graph, the student model, the diagnostic algorithm, the task-selection algorithm, the spaced repetition algorithm, and so on. “AI“ is mentioned only in the headline and the first sentence.

This reinforces my contention that the references to “AI“ and “AI-powered” are mainly marketing fluff and could be dispensed with mostly or even totally. Call it an “expert system” or an “algorithm-driven system,” refer to “advanced machine learning” or “adaptive analytics” embedded in the system, and so on. Just try to limit the use of the term “AI.”

Pricing

Assuming someone is intrigued by the idea of Math Academy, the most salient point for them is likely to be the price: $49 a month is a pretty substantial sum for most people. Math Academy is clearly pricing the service based on its value to a user, not based on the marginal cost to support an incremental new user (which I presume is fairly low). But, at least to me, the price is worth it: I place a substantial personal value on knowing an area of mathematics that has previously eluded me, and I’m happy to pay for a service that (at least thus far) seems a good choice for helping me do that.

The price also serves as a filter to discourage people whose motivation is not that high and for whom Math Academy would not be an effective learning experience. The remaining people who do pay the $49 a month are not only motivated to actually use Math Academy, they’re also motivated to talk up the service to others—if for no other reason than to justify to themselves why they’re paying so much. (This blog series can be seen as an extreme example of that.)

Finally, the high price and the absence of a free option helps fund development and operation of the Math Academy service, helps Math Academy be profitable as soon as possible, and helps ensure that the company and service will continue to be available for the foreseeable future.

However, there are a few ways I can see the pricing scheme being tweaked. The first—already floated by Math Academy itself—is to make the service more affordable to people in countries where typical incomes are a fraction of what they are in the US and similar countries. Due to the deficiencies of their public school systems, students in many of those countries already pay for private tutoring at a much higher rate than US students, and thus would be good prospects for a service that could credibly promise to accelerate their mathematical education.

There will certainly be people in the US and elsewhere who will try to game any geo-based pricing scheme, for example, by using a VPN to make it look like they’re based in India rather than Indiana. However, it’s not clear that this would be a major problem, and there are potential ways to reduce it. (For example, the cheapest prices might only be for access to versions of the service in other languages, say, Hindi or Chinese.)

There are also likely students within the US and similar countries who would be very good candidates for Math Academy but whose families cannot afford it, namely mathematically promising students from disadvantaged backgrounds. Math Academy could offer discounted subscriptions directly to such students, but another possible approach would be to reach out to local groups working to identify and assist mathematically gifted students, and offer them discounted group subscriptions that could be paid for by donors to such groups. Math Academy is apparently also considering offering family discounts, something that would presumably be of great interest to the home-schooling community.

I’ll elaborate more on this in my next post, but I can also see a potential market among adult learners who are very motivated to learn mathematics up to a particular level (equivalent to Mathematical Foundations II or even just Mathematical Foundations I) and are not interested in (or capable of) progressing further. A per-course flat price might be a better approach for that segment of the market, especially when combined with my next idea:

There’s a particular pricing issue that is likely to affect me and perhaps other adult learners: Let’s say that I succeed in my quest to learn what an eigenvector is (i.e., by completing the Math Academy Linear Algebra course). I’ll likely want to go on and take one or two other courses (Multivariable Calculus and Probability and Statistics being the most likely ones), but I doubt that I’d want to continue cranking through every advanced course that Math Academy has to offer.

So, what would I do then? The obvious answer is to cancel my Math Academy subscription. But what if I’m afraid of forgetting what I’ve learned, and don’t have total confidence in previous spaced repetition reviews having foreclosed that possibility? In that case, I’d be interested in switching to a hypothetical reduced-price Math Academy subscription that just provided ongoing reviews of material from previously completed courses, say for $5-10 per month. Math Academy would then keep me as an ongoing customer, and I could always upgrade my subscription back to full price if I happened to find a new course I was interested in taking.

(The Math Academy “test prep” mode—which I haven’t tried—is somewhat reminiscent of this, in that it prevents moving to another course and instead just does reviews for the current one. But this is intended only as a temporary, not permanent, measure, and you’re still paying full price in the meantime.)

User experience

As of today, the only way to use the Math Academy service is as a website. I started out using the Math Academy on my laptop, a 13-inch MacBook Air, and it was perfectly usable in that context. However, after a couple of weeks I switched to using the website primarily on my iPad Mini. By doing this I could easily take the iPad Mini and a similarly sized 5-inch by 8-inch notepad (to work on problems) and work on Math Academy lessons anywhere in the house that was convenient: home office, living room, or bedroom. I could also easily take them and work on problems elsewhere, like in a library or if I were eating in a restaurant by myself.

The present Math Academy website is usable on an iPad Mini or similarly-sized tablets, but the experience is not perfect in all respects. The main problems I encountered were as follows:

First, the small text entry boxes used for some answers are difficult to use unless you really expand the page. This is especially true for the special features used to enter fractions or square roots; after entering part of an answer using them I often found myself struggling to get the cursor to go back to the remainder of the text field, because my finger is so large relative to the input fields. On a desktop PC or laptop the tab key can be used to move back to the enclosing input subfield, but that’s not possible on a tablet (at least, not without the use of an external keyboard). Perhaps an additional UI element could be added that simulates the effect of typing the tab key?

Second, when taking quizzes, the position of the “Prev” and “Next” buttons was very inconvenient when using my iPad Mini in portrait mode, because they were off the screen when viewing the problem itself at a normal readable size. Once I entered an answer I found myself scrolling the page around to try to find the “Next” button to go to the next problem—an annoying thing to have to do when you’re taking a timed quiz and every second counts. Quizzes do work better with a tablet in landscape mode, but that’s a less natural way to hold a tablet if you’re not watching a video.

My final thoughts on the user interface: I think keeping the Math Academy service as a traditional website is perfectly fine. I don’t feel the need to have an iPad app (or a similar app for other tablets), and I certainly don’t feel the need for a Math Academy smartphone app. (For one thing, any non-trivial use of the service requires you to be writing on a scratch pad to work problems, something you can’t do in the typical situations in which you’d use a smartphone, like standing in line at the grocery store.) Maybe having a phone app would be useful for markets in other countries where a smartphone is the only device used by most users, but I don’t see it as mandatory for the US market.

However, I definitely would like to see the website tweaked to be more usable on tablet devices down to the size of an iPad Mini or comparable Android models. This includes relatively larger default fonts, relatively larger text input fields and a way to handle the tab key issue, and a page organization that can fit all needed UI elements within a portrait-mode tablet screen without having to scroll the page horizontally. Due to the issues I’ve encountered, I’ve decided to switch back to my laptop for now.

I’ll have more feedback to offer in part 10 of this series, in which I’ll discuss the learning-related aspects of Math Academy as I experienced them.

As in previous posts, the following sections are my paraphrases of the content of The Math Academy Way; my own comments are [enclosed in square brackets]. You should interpret statements not in brackets as being prefaced by “The book says . . .” or “The author claims that . . .” Terms in boldface are key concepts relevant to the Math Academy system.

Frequently Asked Questions

Many of these are relevant primarily to people already taking Math Academy courses, but there are several exceptions.

First, the FAQ includes an explanation of how course lessons are divided up into instruction and active problem solving, interleaved with each other. More specifically, the lesson begins with an introduction (presented using slides), followed by a worked example covering the material just introduced and 2-3 practice problems similar to the worked example. The worked example plus practice problems is known as a knowledge point (KP), and there are typically 3-4 per lesson.

Failure to complete a KP results in “failing” the lesson, and the student is moved on to different lessons before coming back to the failed one. Math Academy claims an average pass rate of 95% on the first try, and 98% within two tries.

A subsequent question addresses the concern about the large amount of problem solving required of Math Academy students. The response is that only doing active problem solving as soon as possible after learning something will ensure that the information will in fact be committed to long-term memory. [This concern about Math Academy’s relentless focus on problem solving is also reminiscent of Dan Meyer’s complaint that Math Academy, like other proposed instructional metholodogies, is simply “[redefining] math to mean ‘becoming an absolute demon at math exercises.’” More on that later.]

Further questions reiterate core tenets of the Math Academy approach:

It’s not necessary to struggle in order to learn something—and in fact struggling is counterproductive. “The way to increase a student’s ability to make mental leaps is not by having them jump further, but by having them build bridges from which to jump.”

Automaticity is important, and something the Math Academy needs to check for. It is not necessary to achieve full automaticity to advance to higher-level topics, but a lack of automaticity will eventually catch up with students and impede further progress.

Consulting and leveraging worked examples is essential to make continued progress. “If you don’t have worked examples and instructional scaffolding to help carry you through once math becomes hard for you, then every problem basically blows up into a ‘research project’ for you.”

[This point is also relevant to Dan Meyer’s criticisms of the Math Academy approach. He dismisses the fact that Math Academy provides examples to teach concepts: “You don’t get Math Academy ‘experience points’ for reading conceptually rich explanations. You get them for completing exercises.” The counterpoint is that if you don’t read the explanations and examples, and as a consequence lack basic understanding of the concepts, then at some point you’ll no longer be able to successfully complete the exercises, and you’ll stop getting those sweet, sweet XP.]

[For example, Dan Meyer describes teaching his young children to mechanically calculate derivatives: “Easy! All I did was tell them to take the number above the x and write it next to the x, then to subtract one from the number above the x.” His kids could no doubt rack up some XP doing exercises of that type, but their XP acquisition would grind to a halt as soon as they came to a problem that asked them to calculate the slope of a tangent line to a curve.]

Math Academy courses are not structured like typical higher math textbooks because that style of instruction is demonstrably ineffective except for the most talented students. “Higher math textbooks and classes are typically not aligned with (and are often in direct opposition to) decades of research into the cognitive science of learning. Higher math is heavily g-loaded, which creates a cognitive barrier for many students. The goal of guided and scaffolded instruction is to help boost students over that barrier.”

Both interleaving of topics and the difficulty of reviews are designed to promote learning by making the task of retrieving information more effortful: It’s easier to remember something on a quiz or review if you’ve just covered the material, and it’s more satisfying to be able to ace every quiz or review, but that means that any learning may be shallow and the supposedly-learned information easily forgotten.

Learning with the Math Academy system requires that students put in a reasonable level of effort on a continuing basis. “Math Academy teaches math as though we were training a professional athlete or musician, or anyone looking to acquire a skill to the highest degree possible. . . . While it’s true that willingness to work hard is a bottleneck for many students, such students are not part of our target market.”

[This statement will no doubt be seized upon by critics who claim that any success Math Academy might have is due purely to selection effects: they are in effect “skimming off the cream” and leaving all the other students in the lurch. It’s certainly a fair criticism that Math Academy demands a high degree of self-motivation. One might hope that other people might be able to step in to help motivate students who are not as willing to put in the effort, but The Math Academy Way doesn’t spend much time on that issue—the “Coaching” chapter is one of the shortest ones in the book.]

[It’s easy to dismiss criticisms like this—for example, by questioning how much typical public school math teachers are able to motivate their own students who lack motivation. But I think it’s worth thinking about how Math Academy might be extended or supplemented to address this. One can blue-sky high-tech solutions, like an LLM designed to be an encouraging companion to an individual student, or low-tech solutions, like local Math Academy meet-ups where students could meet each other, share experiences, and possibly get special assistance from a volunteer or paid instructor. I’m not sure what might work or what might be feasible, but I think addressing this issue to at least some degree will be key to extending the addressable market for this method of online instruction.]

There are many other questions in the FAQ, but I’ll stop at this point.

Notes for Future Additions

This section contains material that is under consideration for incorporation in the main text of The Math Academy Way, and is interesting as a further look into the thinking of those creating the Math Academy system.

Apropos of my comments in the previous section, there’s a lot of suggestions for extending the “Coaching” chapter. For the most part the suggested additions continue the theme of Math Academy as an equivalent to intensive athletic or musical training, and the focus is almost exclusively on parents motivating their children. [Again, this reinforces the idea that Math Academy is not interested in—or has rationally chosen not to pursue—promoting its service to the traditional public education market.]

There is an interesting discussion of knowledge spaces, which are at least superficially comparable to the knowledge graphs of Math Academy, but proved to be too complex to implement in the context of a Math Academy-like system. [There is a commercial product, ALEKS, based on the knowledge spaces framework. It would be interesting to know more about how successful it has been, particularly since it’s sold by McGraw Hill and pitched to the K-12 education market, among others.]

One other interesting but brief mention is regarding “elaborative interrogration,” that is, asking students to elaborate on their understanding of particular concepts and procedures. [If Math Academy is going to incorporate LLMs in some contexts—and right now there’s no firm indication of this either way—this might be an area where they could be useful, if for no other reason than being able to better interpret free-form answers.]

The section includes with many more links to further reading. These may be of interest to anyone who’s gotten this far in the book and want to explore more material along the same lines.

This concludes my discussion of The Math Academy Way. In part 9 of this series I’ll discuss my own experience as a Math Academy student, starting with those aspects of the service unrelated to the actual learning of mathematics.

As in previous posts, the following sections are my paraphrases of the content of The Math Academy Way; my own comments are [enclosed in square brackets]. You should interpret statements not in brackets as being prefaced by “The book says . . .” or “The author claims that . . .” Terms in boldface are key concepts relevant to the Math Academy system.

Chapter 26. Technical Deep Dive on Space Repetition

This chapter elaborates on the custom spaced repetition algorithm employed in the Math Academy system, Fractional Implicit Repetition (FIRe). The basic idea is that in a hierarchical body of knowledge like mathematics, if a student does a spaced repetition review for one mathematical topic, that implies they are also doing a review of other topics that are the original topic’s prerequisites. For example, doing a review of multiplying a two-digit number by a one-digit number (say, 4 x 12) implies also reviewing multiplying a one-digit number by a one-digit number (4 x 2 = 8) and adding a one-digit number to a two-digit number (40 + 8 = 48).

This allows the Math Academy system to reduce the overall number of spaced repetition reviews presented to the student: a single review is implicitly covering multiple other topics (two in the example), and successfully completing the review should “reward” the student by being reflected in the spaced repetition schedule for those topics.

However, there is a catch: while the review for the original topic was presumably presented to the student according to a schedule optimized for retention (per standard SRS practice), the implicit reviews for the prerequisite topics were not. In particular, those reviews may have occurred earlier than they should have been if the reviews were being scheduled in the normal way, and this may negatively affect retention of that material.

The chosen solution is to give the student only partial credit for successfully completing the (implicit) reviews of the prerequisite topic. [The book does not present the exact algorithm by which this done, but presumably the fractional credit is low (close to zero) if the time of review for a prerequisite topic is well before when that topic would normally be reviewed, and is high (close to one) if the time of review is almost at the point where that topic would normally be reviewed.]

Now consider when the student is scheduled for a review of a prerequisite topic, say adding a one-digit number to a two-digit number, according to the standard spaced repetition schedule. If the student fails that review, then clearly that will negatively impact the spaced repetition schedule for the prerequisite topic, i.e., the system will schedule the next review sooner than otherwise. However, failing the review for the prerequisite is also implicitly a failure of review for the original topic, in this case multiplying a two-digit number by a one-digit number. “If you can’t add a one-digit number to a two-digit number, then there’s no way you’re able to multiply a two-digit number by a one-digit number.” So the schedule for that topic should be penalized as well, but again with appropriate discounting.

As a general statement, successfully completing a scheduled review of a particular topic will positively impact the spaced repetition schedules (i.e., by reducing the number of required reviews) for that topic and for all lower-level topics that are prerequisites for that topic (with appropriate discounting). Likewise, failing to complete a scheduled review of a particular topic will negatively impact the spaced repetition schedules (i.e., by increasing the number of required reviews) for that topic and for all higher-level topics for which that topic is a prerequisite (again, with appropriate discounting). “Visually, credit travels downwards through the knowledge graph like lightning bolts. Penalties travel upwards through the knowledge graph like growing trees.”

There is a further refinement: in some cases reviewing a particular topic doesn’t fully constitute an implicit review of a prerequisite topic, since that review may only partially depend on the prerequisite (partial encompassing). For example, of the problem set used for review of integration by parts, only a few problems (say 20% of them) may involve integrating trigonometric functions. This limits the amount of credit that may be given for an implicit review: a successful review of integration by parts may at most provide 20% of the full credit for a successful review of integrating trig functions; this 0.2 credit would then be further discounted as discussed above according to the review schedule for integration of trig functions.

This can be represented in the knowledge graph as a set of weights: of three prerequisite topics for the example topic of integration by parts, integration of polynomial functions may receive full credit (weight of 1.0), integration of exponential functions may receive half credit (weight of 0.5), and integration of trigonometric functions may receive only 20% of full credit (weight of 0.2).

Now, as noted above, successful review of a topic provides implicit credit not just for that topic’s prerequisites, but also for the topics that are prerequisites to those prerequisites in turn, and so on down the knowledge graph. So in theory a given topic has weights as described above with every topic in the knowledge graph that is an “ancestor” of the original topic. Similarly, a topic has weights with all topics for which it itself is a prerequisite.

For a course with n topics, the number of possible weights is n x (n - 1) / 2. For example, a course with 1,000 individual topics would have (1,000 x 999) / 2 = 499,500 possible weights.

However, in practice implementing the FIRe algorithm does not require that all of these weights be explicitly specified. First, some of them can be inferred. [For example, if topic A has a weight of 0.5 with its direct prerequisite topic B, and topic B has a weight of 0.4 with its own direct prerequisite topic C, then presumably the weight of A with C can be inferred as 0.5 x 0.4 = 0.2.]

Second, if the distance in the knowledge graph between two topics A and Z is large enough then the weight can be assumed to be zero, even if topic A is fully encompassed in topic Z. [I believe this full encompassing corresponds to all the weights on the edges of the nodes between the topics, e.g., A to B, B to C, . . ., Y to Z, being one.] This is because by the time the student starts explicit reviews on the more advanced topic Z they would have already completed most of their explicit reviews of the topic A encountered much earlier in the course. Thus there is no real value in giving topic A any implicit credit resulting from a successful review of topic Z. [Note that the book does not explicitly define what distances are considered to be “large” for the purpose above.]

So as a result the number of weights that must be manually assigned is found to be relatively low. [I may write more about this after I watch the YouTube video about assigning weights.]

The chapter then discusses the case when a topic in one course “encompasses” a lower-level topic in another course, i.e., the first topic presumes knowledge of the material in the second topic, even though the second topic is not formally a prerequisite for the first topic (being in a different course).

This is known as non-ancestor encompassing, and weights are set so that successful review of the higher-level topic in one course provides (full) review credit for the topic in the lower-level course. [Note that this increases the total number of weights that must be assigned.]

The next section discusses “student-topic learning speed,” defined as the ratio of “speedup due to [greater] student ability” to “slowdown due to [greater] topic difficulty.” Thus learning speed would be greatest for a strong student studying easy material, and least for a weak student studying difficult material.

Student ability (relative to a given topic) is measured by looking at the accuracy of their answers for reviews and quizzes for that topic. Student ability is predicted at their beginning of the topic based on prior performance on prerequisites and other relevant material, and then is modified as they answer questions.

Topic difficulty is measured by looking at answers for that topic across all “serious” students [where “serious” is not otherwise defined]. It can be used to help formulate a prediction of student learning speed on a given topic.

The Math Academy spaced repetition formulas

Frank here! The final section of this chapter discusses the Math Academy formulas relating to spaced repetition. This subsection is my commentary on those formulas; due to the length of it I’ve dispensed with enclosing the text in sequare brackets.

The first formula is as follows:

repNum → max(0, repNum + speed · decay ^failed · netWork)

Here repNum is a value representing the amount of successful space repetition review that the student has done for a given topic. The book refers to this as “how many successful repetition rounds a student has accumulated,” but this should not be interpreted as a literal count. Instead it is a value that can be adjusted upward or downward at each review, depending on the factors in the formula. (See also the discussion in chapter 18 in the section “Calibrating to Individual Students and Topics” regarding a review being worth more or less than one spaced repetition.)

The repNum value is tracked for each topic, and is updated at each review, whether that review is an explicit review of that topic or an implicit review of it (i.e., an explicit review of a more advanced topic for which the topic in question is a prerequisite).

The first and most important factor in that calculation is netWork, described as “how much net work the student accomplished during the rewiew.” For an explicit review of a particular topic, netWork is presumably equal to, or at least directly proportional to, the amount of XP the student is granted or penalized as a result of the review; for example, if the student passed the review and was granted 4 XP, netWork would be 4 or some fraction of it. If, on the other hand, the student failed the review and was penalized 2 XP, netWork would be negative, and its magnitude would be half that of the successful review in this example.

For implicit reviews of a topic, netWork would be discounted from the full value, as discussed previously.

The second factor is speed, a value representing the student’s learning speed relative to the assumed typical learning speed. If a student is learning faster than most, speed will be greater than one, and netWork will be multiplied accordingly when calculating the new value of repNum; if they are learning slower than most, speed will be less than one (but still positive).

(See also chapter 18, section “Calibrating to Individual Students and Topics”: “If a student does a review on a topic for which their learning speed is 2x, then that review counts as being worth 2 spaced repetitions. Likewise, if a student does a review on a topic for which their learning speed is 0.5x, then that review counts as being worth 0.5 spaced repetitions.”)

Finally, delay is used to penalize students who have gone a long time since the last review and then failed the current one. The value failed is 0 if the review is successful, in which case we have decay ^failed = decay ⁰ = 1; in other words, there is no delay-related penalty imposed. On the other hand, failed is 1 if the student failed the review, in which case we have decay ^failed = decay ¹ = delay, and the penalty is imposed.

The delay value is a positive value that starts out at 1 but is further increased if the student has gone past the scheduled interval for a review. In that case, if the student fails the review then the (positive) delay value multiplies the (negative) netWork value to reduce the new repNum value beyond what it would have been reduced to if the student had not delayed the review.

The second formula is as follows:

memory → max(0, memory + netWork)(0.5)^{(days/interval)}

Memory to the left of the arrow is a numeric value representing the student’s memory of a topic just prior to doing a spaced repetition review. Immediately after the review memory is assumed to change by an amount netWork. If the review is successful then netWork is positive, representing an increase in the student’s memory of a topic. On the other hand, the netWork value will be negative if the student fails the review, representing a decrease in the student’s memory of a topic. However, memory can never decrease below 0 (representing total forgetting of a topic), so the “max” function is used to ensure that.

Once the review is complete and the student’s memory value is recalculated, it then starts to decay exponentially as time goes on. The speed of the decay is related to the spaced repetition interval as follows: the spaced repetition interval is calculated to be the number of days after the review at which the student’s memory has decayed to half the original value it had immediately after the last review.

Immediately after the review, the days value in days / interval is 0, so we have (0.5)^{days/interval} = (0.5)⁰ = 1; in other words, no memory decay has yet taken place. When the number of days after the review is equal to the calculated spaced repetition interval then we have (0.5)^{days/interval} = (0.5)¹ = 0.5, and memory has decayed to half its original value.

Memory continues decaying if the student goes past the calculated space repetition review interval without doing a review. For example, if the student goes twice the interval period without a review then we have (0.5)^{days/interval} = (0.5)² = 0.25; in other words, memory has decayed to a quarter of its original value.

Chapter 24. Technical Deep Dive on Diagnostic Exams

New students on Math Academy need to take a diagnostic exam before beginning a course, to judge whether the student has mastered topics that are prerequisites for the course. This exam would be unacceptably long if the student needed to be tested on every possible prerequisite, potentially requiring up to a thousand questions.

However, the hierarchical structure of mathematics (as reflected in the Math Academy knowledge graph), along with some other techniques, allows the exam to get acceptable results (in terms of proper placement) with relatively few questions (20-60 depending on the course level). Successful answers for a more advanced question indicate that the student should also be successful answering questions on less advanced prerequisites; thus the system can skip answering those questions.

Success on a question for a given topic can also be correlated with success on a different question on a different topic that is relatively unrelated to the first (neither topic is a prerequisite for the other). That can also allow for the second question to be skipped, instead inferring its result from the result on the first question.

The diagnostic exam also attempts to measure knowledge confidence, that is, whether the Math Academy system can reasonably conclude that the student has the applicable knowledge. If the student successfully answers a more advanced question but fails to answer a simpler question, or if the student takes an unacceptably long time to answer a question, then the system’s confidence in the student’s knowledge will decrease. The system can then compensate by being prepared to go back to earlier material if the student starts having issues on the current material.

In general, the diagnostic exam is conservative in its assessment of a student’s knowledge, to avoid placing the student in a course for which they’re not prepared. In doing actual course work the student will typically be assessed as performing at a somewhat higher level (the “edge of mastery”).

If needed (e.g., due to a change in the knowledge graph), the system can do supplemental diagnostics from time to time to produce a more accurate assessment of the student’s knowledge.

Given the importance of the diagnostic exam and the need to ensure an accurate assessment, diagnostic questions are created manually by Math Academy staff. [As with the knowledge graph, the set of diagnostic questions forms an important component of the overall Math Academy intellectual property portfolio. However, lie the knowledge graph, the questions themselves are publicly visible, and hence can be scraped by competitors.]

Chapter 28. Technical Deep Dive on Learning Efficiency

Learning efficiency is the extent to which a student can complete all spaced repetition reviews without having to explicitly review previously learned material. Efficiency is at its theoretical lowest when all topics are independent of each other and need to be reviewed individually. [A good example is flashcard-based learning of unrelated facts, like the capital cities of the fifty US states.] Efficiency is at its theoretical maximum when each topic is the sole prerequisite for the next, so that reviewing a topic implicitly reviews all its predecessors.

Because of the hierarchical nature of mathematics as reflected in the knowledge graph, in which one topic encompasses many others, learning efficiency in the Math Academy system can be much closer to the theoretical maximum. The empirical result is that on average most courses require only about one explicit review for each topic covered.

The Math Academy gets closer to the theoretical maximum learning efficiency by taking all the repetition reviews due for various topics and “compressing” them: retaining only those that cover all of the topics associated with the due reviews and contribute the most in terms of space repetition reviews across the entire student knowledge profile (repetition compression).

Students can vary in their learning efficiency percentage; for example, an efficiency of 0.5 corresponds to taking twice the expected time to complete all the work for a course. This work is measured in eXperience Points (XP), which represent one minute’s work by an average student who is serious about their studies but does make some mistakes. So a given course will be considered as requiring, say, 3,000 XP. [Translated to time, this 3,000 XP would be about 50 hours, i.e., 3,000 minutes divided by 60 minutes per hour.]

In addition to the quality of a student’s work affecting their learning efficiency (by answering questions correctly and avoiding excessive reviews), devoting more time to studying can also increase learning efficiency, with it being empirically measured to be proportional to the pace of studying raised to the exponent 0.1. Thus doubling the pace (doing twice the amount of studiny per day) increases efficiency by about 2^0.1 = 1.07, a 7% increase.

So, increasing the pace increases learning efficiency, which in turn means it will take less time to complete a course than it would otherwise. However the overall determinant of course completion time is still just how many minutes (XP) one can spend each day. So, for example, doing 40 XP a day (assumed to correspond to a learning efficiency of 1) would allow a student to complete a 3,000 XP course in 75 days or 15 weeks. [This assumes the student studies 5 days a week, as in a typical school or home-schooling environment.]

If the student instead did 160 XP per day (almost 3 hours of work) this would correspond to a pace of 4x normal, their learning efficiency would improve to about 1.15, and the time for course completion would be 3000 / (160 * 1.15) = 16.3 days or just over 3 weeks [again assuming 5 days of work a week].

On the other hand, a pace of 10 XP per day (about 10 minutes) would correspond to a pace 0.25x normal, their learning efficiency would decrease to about 0.87, and the time for course completion would be 3000 / (10 * 0.87) = 345 days or 69 weeks [again assuming 5 days of work a week]. It would thus take the student more than a year to complete the course.

A typical mathematics course takes 36 [5-day] weeks, with 50 minutes of class time and 50 minutes of homework per day. If the student did 100 XP per day on a 3,000-XP Math Academy course, they would complete the course in 5-6 weeks, about a 6x speedup. [This corresponds to a learning efficiency of (100 / 40)^^0.1 = 1.1, and a completion time of 3000 / (100 * 1.1) = 27 days, or about 5 1/2 weeks.]

Math Academy recommends doing at least 15 XP per day to complete a course in a reasonable time (less than a year), but recommends a faster pace for best results. [The calculated learning efficiency for 15 XP per day would be (15 / 40)^^0.1 = 0.91, and the course course completion time for a 3,000 XP course would be 3000 / (15 * 0.91) = 220 days or 44 weeks, a little bit longer than a traditional school course.]

Chapter 29. Technical Deep Dive on Prioritizing Core Topics

A Math Academy course includes both core topics and supplemental topics, with core topics prioritized. Core topics are identified by a proprietary algorithm running against the course’s knowledge graph. Any topic identified as core will have all its prerequisites as core as well.

Supplemental topics are often present mainly because they are part of educational standards (e.g., Common Core). Core topics are the focus of the Mathematical Foundations (MF) series of courses, which are intended for adult learners who need a refresher on K-12 math but are not subject to Common Core or other requirements. The Mathematical Foundations courses are prerequisites for university-level courses.

This concludes my discussion of Part V of The Math Academy Way. In part 8 of this series I’ll discuss the final sections of the book, with a focus on “Frequently Asked Questions,” which includes answers for questions that might be asked by either students taking Math Academy courses or those interested in doing so.

As in previous posts, the following sections are my paraphrases of the content of The Math Academy Way; my own comments are [enclosed in square brackets]. You should interpret statements not in brackets as being prefaced by “The book says . . .” or “The author claims that . . .” Terms in boldface are key concepts relevant to the Math Academy system.

Chapter 24. Parental Support (In Progress)

Because deliberative practice requires effort, students need support and encouragement to do it, and need to be held accountable by responsible others (parents or teachers) when they don’t do it. Otherwise students lose motivation and give up on Math Academy.

[This is the only place thus far where teachers are mentioned as potentially assisting students using the Math Academy system.]

The above is the bare minimum needed. The ideal level of support is similar to that characteristic of the families of competitive musicians (and other high-achieving performers), which include daily supervised practice, ongoing evaluation and discussion of progress, motivation through rewards and encouragement, and so on.

[This chapter does not mention it, but even the bare minimum is going to be hard to maintain in families where work, illness, poverty, single parent, etc., make ongoing support and supervision difficult.]

Chapter 25. In-Task Coaching

The main point emphasized in this incomplete chapter is the negative effect of a student using reference material while completing a task in the Math Academy system. This is because they’re using the reference material as a crutch instead of trying to retrieve the material from memory (effortful retrieval).

[I have occasionally done this while using the Math Academy system, looking back at a prerequisite topic before beginning a lesson on a topic I felt a little uncomfortable taking on, or copying down a formula discussed in the example section of a lesson before embarking on answering the questions. As the book says, though, this is generally not a good idea, and I try to do it only occasionally.]

[I’ve also done something related but a little different: Before beginning a series of questions, I sometimes write out from memory the formulas that I’ll need. Other times, if I can’t remember a formula I’ll try to derive it from other formulas I know. For example, one time I couldn’t remember the formula for the derivative of the tangent function, so I worked it out using the definition of the tangent function (sine divided by cosine), the formula for differentiating the quotient of two functions, and the formulas for the derivatives of the sine and cosine functions. It’s not clear to me whether either of these practices are a violation of the advice in this chapter.]

This concludes my discussion of Part IV of The Math Academy Way. In part 7 of this series I’ll discuss Part V of the book, “Technical deep dives,” which decribes in more detail the “technologies” that support the learning strategies discussed in Part III.

As in my previous posts, the following sections are my paraphrases of the content of The Math Academy Way; my own comments are [enclosed in square brackets]. You should interpret statements not in brackets as being prefaced by “The book says . . .” or “The author claims that . . .” Terms in boldface are key concepts relevant to the Math Academy system.

Chapter 10. Active Learning

This chapter reemphasizes that active learning (doing exercises, etc.) is much more effective than passive learning (watching videos and lectures, reading and re-reading textbooks,etc.). In classroom settings this requires all students to be actively learning individually, not just doing exercises as a group.

Multiple examples show the need for active learning. In the first (hypothetical) example, a personal tennis coach talks about tennis and demonstrates moves, but does not have the student practice them. The obvious result would be no learning. In a second (real-life) example, a system was tested that provided instructors improved feedback for lectures, but it was discovered most students were not paying attention anyway.

A third example is MIT physics courses incorporating active learning and reducing the number of students failing by almost 2/3. Finally, it was discovered that elite skaters spend 6x more practice time vs. rest time on jumps, etc., that they’re trying to master. (The Math Academy system has students spending 7x time doing exercises vs. reading worked examples.)

The neuroscience behind active learning: active learning leads to more brain activity, both during active learning and during subsequent passive learning.

Why are there misconceptions around active learning? Passive learning is more convenient for teachers and less stressful for students—and thus less stressful for teachers in turn. Students think they are learning something when they actually aren’t.

Chapter 11. Direct Instruction (in-progress)

[This is a fragmentary chapter with minimal material thus far.]

We should not reject direct instruction on the basis that active learning with passive guidance is more effective than passive learning plus direct instruction. This tells us nothing about the effectiveness of direct instruction on its own, but rather just demonstrates the superiority of active learning over passive learning: Active + Direct > Active + Unguided > Passive + Direct.

Chapter 12. Deliberate Practice

Deliberate practice is “mindful repetition at the edge of one’s ability,” contrasted with mindless repetition of things one already knows. The chapter includes much discussion about the effectiveness of deliberate practice vs. non-deliberative, as well as the fact that deliberate practice is hard for students, since one must continually repeat things “at the edge of one’s ability.” One can supplement deliberate practice with other more fun things to help motivate students, but these are not an effective substitute for it.

Chapter 13. Mastery Learning

The basic idea here is to require the student to demonstrate proficiency in whatever areas are prerequisites for their next learning challenge. In the limit this requires one-on-one instruction, either by a tutor or by a system like Math Academy.

The idea of mastery learning is resisted by traditional educators because it plays havoc with the traditional grade by grade progression of students.

Math Academy has implemented mastery learning at a very granular level through its “knowledge graph,” discussed above. The knowledge graph dictates what a student can next learn. This is compared to the zone of proximal development, i.e., that set of problems which a student can solve with support but not without it. This corresponds with the knowledge frontier or edge of mastery. The goal is for the student to continually expand that frontier outward.

The knowledge profile is the set of topics in the knowledge graph that the student has already mastered. Placement diagnostics determine what that profile is for a new student, so that they can be presented with material at their knowledge frontier.

Chapter 14, Minimizing Cognitive Load

Math Academy instruction is very fine-grained, with a given area having about 10x the number of steps (e.g., worked example followed by exercise) than typical mathematics curricula. The goal is to minimize cognitive load, i.e., the amount of working memory required to complete a task. This helps prevent students from getting stuck on a particular point in the progression of the curriculum.

Each topic is divided into several knowledge points, each consisting of a worked example plus practice exercises. The example given is adding two digit whole numbers. The first knowledge point might be adding two digit numbers where carrying is not required, e.g., 63 + 12. The second knowledge point might be adding two digit numbers with carrying, e.g., 63 + 18. The third knowledge point might be adding two digit numbers with carrying into the hundreds place, e.g., 63 + 38. And so on. Each knowledge point has a subgoal label, e.g., “adding two digit numbers with carrying.”

Knowledge points also can contain diagrams for visualization in addition to verbal explanations, to leverage dual coding and distribute mental processing between visual processing (visuo-spatial sketchpad) and verbal/audio processing (phonological loop). For higher-level topics this can include flow charts.

As students learn the material, scaffolding is removed: a review question may call upon knowledge from one or two of many different worked examples, and timed quizzes further test knowledge in a context where it is not possible to “look at the book.”

Chapter 15. Developing Automaticity

“Automaticity is the ability to perform low-level skills without conscious effort. [emphasis added]” An analogy is to athletes who can perform low-level skills (dribbling a basketball) while thinking consciously about higher-level game strategies. [A similar analogy would be musicians who can perform low-level tasks of playing an instrument while thinking about higher-level tasks like playing expressively or in a certain style.]

One can achieve automaticity (e.g., in mathematics) by leveraging long-term memory to relieve pressure on the limited capacity of short-term memory. Long-term memory thus becomes an extension of short-term memory that a person can draw upon at will.

Automaticity goes beyond familiarity, and requires accessing learned knowledge quickly and accurately. This is required as a necessary foundation before learning more advanced topics that depend on that knowledge already being learned.

Examples of automaticity and the lack thereof: Students are being taught how to compute cubes as the number multiplied by itself and then multiplied by itself again, for example, 4³ = 4 x 4 x 4. The first student knows 4 x 4 = 16 (from having the multiplication table in long-term memory) and then can apply the learned procedure for multiplying a two digit number by a one digit number. The second student doesn’t know the multiplication table, so needs to compute 4 * 4 as 4 + 4 + 4 + 4 = 16. The third student doesn’t even know the addition fact 4 + 4 = 8, but must count up from 4 by 1 four times: 4 + 1 = 5, 5 + 1 = 6, 6 + 1 = 7, 7 + 1 = 8. This results in the second two students making mistakes in their calculations, requiring additional teacher time to correct their understanding and causing student frustration that they’re not “good at math.”

“Automaticity is a necessary component of creativity.” An example is writing: if a person has difficulty with basic issues of spelling and grammar, they will have difficulty in expressing themselves in a creative way. [This is especially true with skills like writing and doing mathematics that—unlike oral language learning—do not come naturally to students based on their having inborn capabilities.]

Automaticity is also necessary for higher-level thinking, and automaticity in knowing and recalling standard mathematical facts is critical to achieving mathematical literacy and academic success in mathematics.

Finally, the chapter discusses the neuroscience underlying automaticity, the idea that it prevents disruptions to background thinking (the “default mode network”), disruptions that reduce the amount of attention and thought a person can devote to a higher-level task.

Chapter 16. Layering

“Layering is the act of building on top of existing knowledge. [emphasis added]” Layering promotes retroactive facilitation (solving a problem using existing knowledge reinforces memory of that knowledge) and proactive facilitation (knowledge acquired in solving previous problems improving knowledge acquisition needed in solving new problems).

Layering also improves the structural integrity of a person’s acquired knowledge, i.e., having that knowledge not have holes where understanding is lacking.

Math Academy promotes layering by having mastery of one topic lead directly into a new topic, and by leveraging a complete and comprehensive knowledge graph in which all new topics depend and build on previous topics. It also uses additional techniques to promote connections between topics, like presenting multi-part problems requiring knowledge of many previous topics to solve.

A key principle: “Any lesson should cover all types of problems that a student could reasonably be expected to solve if they truly know the topic.” Some other approaches violate this by, for example, presenting calculus in a way that does not require algebra.

Chapter 17. Non-Interference

Learning two related topics at the same time (or close together) can inhibit learning of both (associative interference). The Math Academy system avoids this by spacing related topics out in time, and presenting students with a choice of unrelated next topics, thus achieving non-interference. In addition to promoting learning, this also keeps students interested by increasing variety and reducing unnecessary repetition.

Chapter 18. Spaced Repetition (Distributed Practice)

This chapter reviews conventional information about spaced repetition: that by spacing review out in time, students can mitigate the effect of memory decay (the forgetting curve) and (ideally) retain information indefinitely.

The book criticizes traditional educational practices for neglecting the effectiveness of spaced repetition, and thereby leading students to forget information once they have been tested on it, reducing the amount of information retained by them.

Math Academy has found a way to improve on traditional spaced repetition methods based on flashcards, using fractional implicit repetition (FIRe). In a hierarchical body of knowledge like mathematics, by reviewing a given topic the student is implicitly also reviewing those topics on which the original topic depends; this must be taken into account when constructing a review schedule for a student. The techniques by which this is done have been refined over many years by Math Academy [and therefore form part of its proprietary advantage].

Spaced repetition also promotes generalization: that by reviewing material on a suitable schedule, the student can discover new connections between the topic being reviewed and other topics, and therefore can better transfer their knowledge to related but different topics.

What about the objection that spaced repetition requires reviewing a very large number of items during each review session? Because mathematics is a hierarchical body of knowledge (see above), more advanced skills encompass many more basic skills. Thus the number of reviews can be reduced (repetition compression) by reviewing the advanced skill, which also serves as a review of the basic skills. The example given is that of multiplying a 2-digit number by a single-digit number: reviewing this also reviews multiplying a single-digit number by another single-digit number, as well as adding a single-digit number to a 2-digit number.

However, this cannot always be done. If a student’s learning speed is below average, the Math Academy system will not do implicit reviews but will drop back to doing explicit reviews of more basic material. In this case a review counts as a fraction of a spaced repetition. Conversely, if a student’s learning speed is above average, each review will count as more than one spaced repetition. The Math Academy system computes student-topic learning speeds for each individua student in order to do this effectively.

Spaced repetition can be contrasted with the spiral approach, where an instructor periodically revisits material previously covered. Spiraling amounts to spaced repetition with a fixed schedule, and is less effective than actual spaced repetition. However, it is easier for instructors to implement; true spaced repetition with individualized schedules requires supporting technology like that found in the Math Academy system.

Chapter 19. Interleaving (Mixed Practice)

Interleaving or mixed practice—mixing up exercises on different topics in a single practice session—is contrasted with blocking or blocked practice—doing a bunch of similar exercises on the same topic. [“Blocking” and “blocked” are here used in the sense of doing a homogeneous “block” of exercises. This was what I was doing by systematically doing linear algebra exercises one by one in the order presented in the textbook I was using.]

Interleaving is more effective for a variety of reasons. First, it is more efficient: blocking leads to diminishing returns as the number of similar exercises increases. Second, it helps students better match problem solving techniques to problems, especially when the technique needed is not obvious from the statement of the problem. With blocking, students end up reusing the same technique from problem to problem and can get lost when a different type of problem is posed.

However, blocking can appear to be more effective, and to some degree can be more effective when first learning a skill. This makes it attractive to both students and teachers (who are motivated by the appearance of rapid learning). However, interleaving is more effective for long-term retention of material, as has been experimentally demonstrated. It involves desirable difficulties, i.e., difficulties that promote learning.

Interleaving can occur at two levels, and the Math Academy system features both:

Macro-interleaving is done at the level of topics: the student will be presented with a variety of different topics, as opposed to working on the same topic for an extended period of time. Micro-interleaving is done at the level of review, mixing up problems from different topics.

However, there is a trade-off here, in that it would take an excessively long time to fully interleave all review exercises for a topic (i.e., with exercises for several other topics) before featuring them on quizzes on that topic. So the Math Academy system compromises by blocking exercises during lessons and counting them toward spaced repetition credit.

Chapter 20. The Testing Effect (Retrieval Practice)

The best way to review material is not to re-read it (or re-watch it, or re-listen to it), it’s to be quizzed on it. (This is referred to as the testing effect or retrieval practice effect.) The act of retrieving material helps fix it in long-term memory. This is especially effective when quizzes are combined with spaced repetition.

Frequent quizzes are not used in traditional educational settings as much as they might be; a more typical pattern is to have one mid-term exam and one final exam. However, the Math Academy system does quick quizzes very frequently (continuous assessment), and also does evaluation as part of spaced repetition review.

The Math Academy systems tries to mitigate test-induced anxiety by doing quick quizzes on material the student is already ready to be tested on. Timed tests should not be introduced too early and tests in general should be matched to the student’s current level of proficiency, thus preventing desirable difficulties from turning into undesirable difficulties. Lower proficiency can lead to math anxiety as students find themselves unprepared for tests and do poorly on them.

Doing frequent quizzes helps build self-confidence and prepares the student for more extensive timed testing. Topics show up on low-stakes non-timed quizzes first, with opportunities to retake quizzes and go back to review material. Timed tests on the same topics are done only after the student has demonstrated proficiency on those topics.

Chapter 21. Targeted Remediation

The Math Academy system helps students struggling with certain topics (or certain components within a given topic) not by trying to lower the difficulty of the student’s tasks, e.g., by providing extra feedback and hints (“adaptive feedback”), but rather by giving them additional time and practice on exactly those topics (or components within topics) that are causing them the most difficulty (targeted remediation).

“Targeted remediation at Math Academy’s level of granularity (individual students on individual topics) and integrity (maintaining the bar for success) has not been studied in academic literature. [emphasis in the original]”

Corrective remedial support is tailored to the specific circumstances: providing more questions if a student is struggling with a task, switching them to unrelated topics if they fail a topic before coming back to the original topic, and providing remedial reviews if they appear to be stuck at a particular place. Note that remedial reviews may be for a topic that’s some distance back in the knowledge graph hierarchy, but which is a prerequisite for the current topic. For example, in the topic of cubing a number, a student may have problems with calculating (-4)³ = (-4) x (-4) x (-4) because they have unremediated problems with multiplying negative numbers.

Preventative remediation occurs when the student’s learning speed for a topic is predicted to be low based on their learning speed for other related topics. In this case the Math Academy system can prevent problems by scheduling additional reviews.

Foundational remediation occurs when students start a Math Academy course with holes in their knowledge of the foundational topics for the course. For example, they may not have mastered some topics in arithmetic needed for algebra. In this case the Math Academy system can let them proceed with topics that don’t depend on the unmastered foundational topics, and go back and remediate those unmastered topics when needed.

Finally, the Math Academy developers track student learning to see if there are any topics that students are having particular struggles with. They can then do content remediation, for example by providing additional worked examples or review points within the topic, or by splitting it up into multiple separate topics.

Chapter 22. Gamification

Gamification can improve both student learning and enjoyment if it is properly aligned with education objectives and student motivations and is designed to prevent students gaming the system.

The Math Academy system uses eXperience Points (XP) tied to completion of tasks, each XP representing a minute of sustained effort by an average student. (Optional) competitive weekly leaderboards keep track of students’ activity versus other students of comparable ability. Students accumulating lots of XP get promoted into higher leagues, students not doing so much get relegated to lower divisions [as in the English Premier League].

Students can earn extra XP with perfect performances on tasks, earn little or no XP for nearly passable or poor performance, and get penalized with negative XP if they are perceived to be blowing off tasks.

Student progress is measured separately from XP, based on the percentage of a course that they’ve completed. Progress slows down near the end of a course due to the need to review material from earlier in the course—but the system will never let lesson time (vs. review time) fall below 25% of the total time. There is no leaderboard or other gamification for course progress.

Chapter 23. Leveraging Cognitive Learning Strategies Requires Technology

Teachers are reluctant to implement educational strategies like those embodied in the Math Academy system, but not through any fault of their own. They are working under structural constraints that make it difficult to adopt such strategies, for example, the system of grade-to-grade progression. Even if they could adopt some or all of the strategies then it would be physically impossible to implement them beyond a 1-on-1 tutoring scenario, because students have differing knowledge profiles and learn at different speeds. Thus implementing these strategies via technology is the only possible solution.

[This raises a question: How did this problem play out in the original Math Academy program in Pasadena, the one from which the Math Academy learning system arose? I think this chapter would benefit from a more in-depth treatment of that story, including an account of how it motivated creation of the Math Academy system.]

This concludes my discussion of Part III of The Math Academy Way. In part 6 of this series I’ll discuss Part IV of the book, “Coaching,” a relatively short and incomplete section that discusses how parents can best help children using the Math Academy system, as well as how students can help themselves.

As in my summary of Part I, the following sections are my paraphrases of the content of The Math Academy Way; my own comments are [enclosed in square brackets]. You should interpret statements not in brackets as being prefaced by “The book says . . .” or “The author claims that . . .” Terms in boldface are key concepts relevant to the Math Academy system.

Chapter 6. The persistence of neuromyths

This is a brief chapter that makes the point that laypeople and even experts believe things about the brain and learning that are demonstrably not true. The so-called “Mozart Effect” is given as an example. Belief in such neuromyths is common among those seeking to excuse education failures or looking for a “quick fix” that doesn’t require a lot of effort.

Chapter 7. Myths and Realities about Individual Differences

This chapter begins by pointing out the the idea of different ”learning styles” is a neuromyth: students may prefer receiving information in different ways (e.g., verbal vs. visual), but that does not affect the rate at which they actually learn.

A major factor in how people learn is working memory capacity (WMC): Larger WMC makes tasks easier (all other things being equal), improves the ability to do abstract thinking (which in turn affects the ability to apply learning to new contexts), and improves the speed at which people can learn.

It is not possible to improve WMC. [Although the author does not mention it by name, so-called ”dual n-back” training would seem to be an example of a technique that purports to do this. But see Gwern Branwen’s discussion of whether this actually affects IQ.] However WMC can be augmented by long-term memory that encodes domain-specific knowledge. [A trivial example: avoiding the need to multiple 4 times 16 because one has memorized the answer.] Since people are able to add more information to long-term memory (within limits), this is consistent with the ideas behind growth mindset.

In the context of mathematics, the field can be divided into roughly six levels, from basic arithmetic to Fields medal-level mathematical work. [I was probably at or close to level 4 upon graduating from college—capable of doing graduate-level work at least to some minimal level—and am now below level 3.]

People generally hit an abstraction ceiling, a point at which the time and effort required to learn math to a given level exponentially increases to a point where continuing is not a productive use of one’s time even given sufficient motivation. For people with lower WMC the exponential curve is steeper and they will hit the ceiling sooner, because it takes them more energy to reach a given level of proficiency.

However, if learning can be made more efficient, so that it requires less energy, the exponential curve flattens somewhat and students can reach levels of proficiency they might not otherwise be able to.

WMC and similar capacities have a genetic component, and we cannot assume everybody to be at the same level. Genetic-influenced ability interacts with environmental factors—access to instructors, amount of practice, etc.—to determine the overall level a person can reach in math. Instructors downplay this fact, partly to encourage students and partly for self-interested reasons, because they don’t want to lose students. [Does this same incentive affect Math Academy itself, and, if so, how?]

But people can generally learn more math than they do, and struggling in math is not really an indication of how much one can learn. For example, the struggle may be because a student didn’t master previous material and is not given the opportunity to remedy that. One of Math Academy’s claims is that they can detect and remediate such knowledge gaps with the aid of an ”adaptive, automated learning system.”

Math Academy also avoids such gaps in the first place via a combination of mastery learning, spaced repetition, and comprehensive coverage of all required topics. [See the chapter on the ”knowledge graph.”]

Struggle can also be caused by ineffective practice and insufficient practice. Math Academy can help here by providing suitable practice exercises to encourage active learning, but success is ultimately up to the student’s motivation to put in the work. Such motivation may be intrinsic (the student loves math) or extrinsic (the student needs math for a job, or is rewarded by competitive success or by parents).

Final discussion: If suitable instructional scaffolding, guidance, etc., can compensate for lower WMC and related issues, why couldn’t we employ this to educate all students at the same rate and to the same level?

But in practice some students can ”eat” bigger ”bites” of new material than others can, and this will in practice cause them to progress faster. This is consistent with the observation that math will become difficult for different students at different times in their study.

[This discussion reminds me of Nathan Robinson’s argument that we can’t conclude that hereditary differences in intelligence between individuals exist until/unless we spend many many years trying to teach someone something: “When we have given students a boundlessly kind, supportive engineering program, that lasts as long as they need and is structured around them with as many of society’s resources as possible put toward its perfection . . . then maybe we will know their ‘natural capacity’ for engineering.” The counter-argument in The Math Academy Way is that any given student’s “innate capacity” for mathematics is significantly higher than we suppose, but that students do vary in that capacity, such that some will “hit a wall” earlier than others even in an optimally-structured program.]

Chapter 8. Myths and Realities about Effective Practice

The first part of this chapter promotes direct instruction and criticizes constructivist instruction, discovery learning, and related approaches. One key point is that discovery learning is much more effective for experts who have pre-existing knowledge that they can use as a basis for further exploration, and also more effective in a work setting where no one person is expected to know everything and work output is often a group effort.

But these are not typically the case in traditional education: we want individuals to learn, not groups, and we don’t want student to have the illusion of learning when in reality they’re being carried by other people in the group.

The discussion then switches to the point that no learning is effortless, and learning requires a sustained effort at deliberative practice, i.e., ”individualized training activities specifically chosen to improve specific aspects of a student’s performance.” This can include practice testing and distributed practice (spaced repetition). “Learning is all about creating desirable difficulties.”

Finally, the discussion defends testing, repetition, competition, and (in the context of math) computation from those who feel they detract from learning. The main point here is that testing, repetition, and competition are key to talent development in other fields, and equally if not more so in math. As for computation, it is needed as a basic skill to help build further conceptual understanding.

Chapter 9. Myths and Realities About Mathematical Acceleration

This chapter discusses extensive research that shows that having qualified students take above-grade math courses has many positive benefits and no drawbacks. Students who have mastered prerequisites suffer no negative psychological effects, learn more material to the same depth (as they would without acceleration), don’t run out of math courses (many such courses beyond calculus), get better experience than with math competitions, can place out of college courses (possibly with special appeals to instructors), and better prepare themselves for advanced instruction in math or related fields (e.g., engineering).

Speculations on why schools don’t support acceleration include: It doesn’t fit the traditional one-grade-at-a-time model, especially at breakpoints between elementary and junior high school, and between junior high school and senior high school. Acceleration can also negatively impact school funding because students spend less time in school. [Another possible reason is apparent disparate impact, based on which students get admitted to accelerated classes and which don’t.]

This concludes my discussion of Part II of The Math Academy Way. In part 5 of this series I’ll discuss Part III of the book, which goes into more depth regarding the “features” of the Math Academy system, i.e., the learning strategies it implements.

In this post I turn the floor over to Math Academy itself, summarizing the main arguments of Part I of the book The Math Academy Way: Using the Power of Science to Supercharge Student Learning, by Justin Skycak (advised by Jason Roberts). (The book is still in draft form; I read, took notes on, and am referencing the draft as of December 27, 2024. However, at the time of writing there is a newer draft, dated January 28, 2025.)

The preface of the book lays out the questions the book aims to answer:

1. What techniques exist to maximize student learning and talent development, particularly in the context of math?

2. Why are these techniques so impactful, and if they are indeed so impactful, then why are they so often absent from traditional classrooms?

3. How does Math Academy leverage these techniques?

The book is aimed at pretty much anyone who might be interested in the Math Academy system, not excepting math hobbyists (like me).

The first set of chapters (Part I: Preliminaries) can be thought of as an extended “sales pitch” for Math Academy: identifying the problem and explaining how Math Academy is uniquely positioned to solve it. These and later chapters include copious quotations from and citations of the research literature relevant to Math Academy; as noted in the preface, “when faced with the decision to (a) build credibility by quoting the literature extensively, versus (b) streamline our communication, we have chosen to lean towards credibility.”

The following sections are my paraphrases of the content of The Math Academy Way; my own comments are [enclosed in square brackets]. You should interpret statements not in brackets as being prefaced by “The book says that . . .” or “The author claims that . . .” Terms in boldface are key concepts relevant to the Math Academy system.

Chapter 1. The Two-Sigma Solution

The first chapter riffs off the key claim of education researcher Benjamin Bloom: that one-on-one instruction can elevate a typical student to the 98th percentile, an up to two standard deviation improvement (hence the “two sigma solution”). [However, the book notes that this level of improvement is not seen by other researchers, and José Luis Ricón’s analysis echoes this skepticism.] Leaving aside the issue of exactly how effective it is, Bloom’s method would be extremely costly to do with human instructors even if we wanted to do it: more than $10,000 a year.

But in any case traditional schooling is at odds with the idea of talent development as it is practiced in other areas (e.g., sports and music). The key contrasts are grouping by ability vs. grouping by age, and short-term teacher involvement with a given group of students vs. long-term involvement (cross-sectional vs. longitudinal). Mixing these modes does not seem to be effective.

Talent development itself proceeds in three phases: an early phase focused on having fun learning, a middle phase focused on intensive skill development, and a final phase focused on application to new and original problems at the highest level. Acquiring knowledge and solving problems (the second phase) precedes creative endeavors (the third phase). Math Academy is focused on the second phase and assumes a willingness to learn on the part of the student.

The chapter concludes with a list of citations of relevant research papers, a practice repeated in subsequent chapters.

[To the extent that this book is a sales pitch—and I think in large part it is, whether the author acknowledges it or not—this chapter is in my opinion a missed opportunity. It’s a classic example of “feature-benefit” selling: “here is a feature, and here is the benefit it provides you, the prospective customer.” But this doesn’t address a critical point: why should I or anyone else care? Selling is much more effective when the customer has “pain” that the product can potentially relieve.]

[That’s why I led my previous post with the anecdote about me not knowing what an eigenvector is, goofy though it may be. It is a source of embarrassment and frustration to me that I still haven’t learned a key concept in an area of mathematics I’m interested in, and that embarrassment and frustration are intense enough that I was motivated to read this book and to consider spending $49 a month. So, Math Academy folks, please consider surfacing more pain in your prospects! And not general free-floating pain, but pain as it is specifically felt by students, parents, teachers, adults in the workforce, etc.]

Chapter 2. The Science of Learning

The key message of this chapter is that researchers have empirically demonstrated the key elements related to effective learning: active learning, deliberative practice, mastery learning, minimizing cognitive load, developing automaticity, layering, non-interference, spaced repetition (distributed practice), interleaving (mixed practice), the testing effect (retrieval practice), and gamification.

However, with minor exceptions [like the Math Academy], educators have not incorporated these finding into their teaching practice. The main reason seems to be that using them makes both teachers and students feel that students are not learning fast enough, and both teachers and students value the perception of learning over the actuality: illusion of comprehension prioritized over desirable difficulty.

[Although the book doesn’t mention it, this is true of parents as well: many value good grades more than actual learning.]

Technology can help here, by making it possible for teachers to implement techniques that are too time-consuming to do manually, e.g., creating personalized spaced repetition schedules for students. [But, technology is no panacea here, since effective use of spaced repetition requires a fair amount of discipline on the part of the student.]

The chapter concludes with the claim that by using these techniques the Math Academy system can accelerate learning by 4x.

Chapter 3. Core Science: How the Brain Works

This chapter begins by reviewing the distinction between sensory memory, short-term memory (working memory), and long-term memory. Proper learning techniques can compensate for lower short-term memory.

Solving a math problem (like calculating the value of 4³ ) is a coordinated effort between sensory, short-term, and long-term memory: Sensory memory is used for initial understanding of the problem (calculate 4³ ) and for concrete storage of intermediate results (e.g., writing down “4 * 16” after having done the initial multiplication of 4 * 4). Long-term memory is used for retrieval of memorized facts (e.g., 4 * 4 = 16) and memorized procedures (e.g., how to multiple a two-digit number). Working memory retrieves information from sensory memory and long-term memory, does calculations as needed, and stores the results in sensory memory (as intermediate results) or long-term memory (as final results).

The author notes that items stored in long-term memory can reduce the work done by short-term memory, e.g., memorizing the values of 2³ , 3³ , 4³ , 5³ , etc.

[An alternate—though less general—possibility along the same lines is being familar with “computer arithmetic” and recalling that 16 * 4 = 64. Or, a variant: recasting 4³ as (2² )³ = 2⁶ based on rules involving addition of exponents, and then using the memorized values of powers of two to produce 2⁶ = 64.]

Chapter 4. Core Technology: The Knowledge Graph

This chapter introduces the idea of a knowledge graph, a [directed acyclic] graph showing which (detailed) topics are prerequisites for other (detailed) topics. Some topics may be prerequisites for more than one topic, and some topics may have multiple prerequisites.

[The knowledge graph is analogous to a “tech tree” in Sid Meier’s Civilization and similar games, in which certain technologies must be discovered first in order to enable discovery of others. It’s a directed graph because the edges go in a single direction from a more basic prerequisite topic to a more advanced topic that depends on that prerequisite. It’s an acyclic graph because there are no circular dependencies, in which one topic is a prequisite for another topic, which in turn is a prerequisite for the first.]

The full knowledge graph for Math Academy consists of thousands of topics [nodes], covering mathematics from elementary school to college. Courses are simply subsets of the overall knowledge graph, typically containing a few hundred topics; these topics can then be combined into a course graph.

[The construction of the overall knowledge graph must have taken a lot of work. The graph is thus a possible source of competitive advantage for Math Academy. However, since it’s exposed in the course pages an unscrupulous competitor could presumably copy the entire graph, either directly or—more sneakily—by enrolling an LLM as a student and training it on the course material.]

Mastery learning: Students must demonstrate proficiency in all prerequisites for a topic before being allowed to move on to that topic. Mastery of a topic thus unlocks new portions of the knowledge graph into which students can advance.

A topic can encompass other topics that are prerequisites to it. This simplifies review of material: a student reviewing a given topic does not have to do detailed review of that topic’s prerequisites.

[The book discusses this point in much more detail later. For now, note that this encompassing is made possible by the hierarchical nature of mathematics, in which more advanced areas subsume basic topics. It would not necessarily be possible with other subjects. Also, many people like me who’ve tried learning topics using spaced repetition have felt burdened by the sheer number of items to be reviewed each day, especially when learning a new set of facts. Greatly reducing the number of review items can thus be a major advantage for Math Academy.]

Finally, a new Math Academy student takes a diagnostic exam to determine which topics in the knowledge graph they have mastered and which they have not. This enables them to skip topics they already know in favor of addressing foundational topics that they have not yet mastered.

Chapter 5. Accountability and Incentives

Maximizing learning is difficult and at odds with other possible goals: enjoyment, ease of practice, etc. Doing it successfully requires making decisions on pedagogy, etc., based on how those affect measurable learning. However students and teachers (as well as parents and sometimes employers) resist this approach (which is admittedly not easy to implement).

Accountability is lacking in traditional education because it is diffuse, spread out over multiple teachers: a teacher may be ineffective, and will leave it to the teacher of the next grade to remedy any student deficiencies. [This is of course a consequence of the way education is traditionally structured.]

Grade inflation is rampant. This means that grades as a measure of learning cannot be trusted and can set students up for failure in future courses. Experience during COVID-19 demonstrates this, as math grades inflated and remained inflated after the main part of the epidemic was over. But COVID-19 just accelerated an existing trend.

One response to criticism of grade inflation is to deny the existence of objective learning (radical constructivism), making the student the judge of their own learning. [Although the author does not mention it, this can seen as an example of the general postmodernist approach.] This denial also can be seen as a response to concerns over disparate impact, both during COVID-19 and otherwise. [It’s interesting to speculate whether radical constructivism would have gained a following if such concerns were absent.]

In contrast, Math Academy is held accountable for learning by those who pay for it and [it is implied] exert the discipline to complete the course, and must therefore ensure it employs effective learning strategies. This is in contrast to free or “freemium” offerings that cannot afford to turn away students unwilling to put in serious work.

[Note that charging in and of itself is necessary but not sufficient: There are lots of education offerings that charge a lot of money and purport to deliver actual learning. The key element has to be independent validation of learning by a third party mechanism, e.g., standardized tests or class grades.]

This concludes my discussion of Part I of The Math Academy Way. In part 4 of this series I’ll discuss Part II of the book, which can be thought of as addressing prospects’ objections to the sales pitch of Part I.

But before I do that, let me take a few moments to throw some cold water on Math Academy. Why I am doing this? I’ve spent almost my entire career working in sales groups selling software and hardware to large enterprises. Based on my experience, it’s best for both customer and vendor if a prospect starts the “sales journey” with a healthy amount of skepticism.

It’s too easy to let yourself be overly-dazzled by the promises of new “gee-whiz” technologies, or to be so suffering from painful problems that you leap to adopt the first product that comes along and promises to relieve that pain. That in turn can produce unhappy customers and a vendor distracted by trying to mollify them.

I tried to apply that way of thinking to my own case, doing research and looking for reviews of Math Academy (especially negative ones) rather than instantly reaching for my credit card and signing up for the service as soon as I read about it. So, please allow me to role-play as a skeptical and pessimistic prospective customer of Math Academy, at least for this post. Let’s start:

If you were a betting person and you were coming into this cold, you would bet that Math Academy is not going to “revolutionize education,” is not a “game changer,” and certainly will not have “civilizational impact”—to quote just a few of the things people have written about it.

Why is that? First, your prior position should be to hold as true the “null hypothesis in education,” as the economist Arnold Kling refers to it: that no education intervention—including online education technologies like the Math Academy system—will meet all of the following four tests:

• It is experimentally validated as making a real difference (not just due to selection effects).
• It is persistent (learning does not fade with time).
• It is replicable by groups other than the original educators.
• It is scalable to be able to support large numbers of students.

(The issue of selection effects is particularly relevant to Math Academy, since it appears to be attracting a customer base that is relatively more affluent, knowledgeable about mathematics, and motivated to learn than the student population as a whole.)

Audrey Watters, a critic of education technology, has her own version of Kling’s “null hypothesis”:

A friendly reminder that a meta-analysis of one hundred years of research on ed-tech looks something like this: some students showed some improvement on a standardized test in a specific subject area, after using ed-tech in a class taught by a supportive educator well-trained in that subject area and in the technology in question.

In other words, that’s about all that we can expect of the Math Academy system as well, a system whose roots lie in a context very much like this (a special mathematics program in the Pasadena, California, public school system).

Second, Math Academy touts itself as being based on the “two sigma problem” (or, more optimistically, “two sigma solution”) work promoted by education researcher Benjamin Bloom: that certain types of education practice can elevate a student from the 50th percentile in rank to the 98th percentile, a jump of about two standard deviations (hence the “two sigma”). But apparently other researchers have not been able to fully replicate Bloom’s results, and reading José Luis Ricón’s discussion of “two sigma” research, including some of the techniques touted by Math Academy, left me fairly lukewarm about the possibility of true two sigma solutions, or even one or one-half sigma solutions.

Next, Math Academy the company is yet another in a long line of for-profit and non-profit organizations commercializing online education technologies enthusiastically hyped as revolutionizing education. Your a priori assumption should be that it will fail to live up to that hype, just like its predecessors. Those failures have been exhaustively documented by Audrey Watters; see for example her comments on “The 100 Worst Ed-Tech Debacles of the Decade” (that decade being the 2010s).

Finally, there are various reasons why Math Academy is unlikely to have much impact on education as a whole, at least in the US, where the public school and higher education systems are characterized by a set of entrenched practices and often misaligned incentives. Many of these reasons are discussed in The Math Academy Way, and I’ll cover them in due time. However, they clearly limit the market that Math Academy can address, at least in the near term.

Where then might Math Academy find some traction? There are three obvious markets:

• Homeschooled students, whose parents and guardians have rejected the public school system and are thus not restricted by public school ways of teaching.
• Mathematically talented students (high 90s percentiles), who again are not often finding what they need in the traditional education system.
• Adult learners (like me) who are looking to refresh and recover their mathematical knowledge or learn new areas of mathematics, whether in support of career goals or simply as a hobby.

These are not trivial markets. For example, the National Home Education Research Institute claims there were over 3 million US home-schooled students in 2021-2022. There are over 100,000 US students taking AP Calculus BC (equivalent to a university calculus course). There are also millions of people who subscribe to math-centric video channels like 3Blue1Brown.

However, the cost of Math Academy and the level of student motivation required to complete its courses will likely combine to limit its appeal among those three groups. There may be enough takers to support a sustainable business—50,000 customers at $500-600 a year would be a $25-30 million revenue opportunity—but it’s not clear that there’s enough of a market to achieve the broader ambitions of Math Academy’s founders and staff, much less to have “civilizational impact.”

So, how might Math Academy fail? Let me count the ways:

• The system could fail to meet its education goals, such that students going through the program don’t show much actual improvement in test scores and grades.
• The company could fail to meet its financial goals, not attracting enough customers to keep it solvent and then going out of business.
• The company could be taken over by a larger company that proceeds to “enshittify” the service: for example, turning it into a free service with ads to attract more customers, and then optimizing for “engagement” (and consequent ad viewing) rather than learning.

Math Academy is apparently both self-funded and profitable, and its founders plan to keep it that way. But “everybody has a plan until they get punched in the mouth,” and we can’t rule out the possibility that Math Academy will get “punched in the mouth” financially or otherwise and will have to look to outside investors or even an acquirer for help.

However, the most likely scenario is that Math Academy will remain a profitable company, but will never break out of its initial niches of motivated and relatively affluent home-schooled students, mathematically-talented students, and math-interested adult learners. Most people would count that as a success in business terms, but as noted above I doubt it would satisfy the Math Academy founders and staff, who presumably want to change the entire field of mathematics education for the better.

But those concerns are for the future, which remains unwritten. In the meantime, I still don’t know what an eigenvector is and would very much like to. So, on to part 3, in which I consider the Math Academy sales pitch as embodied in the book The Math Academy Way.

This all started because I don’t know what an eigenvector is. If I were a typical person, that wouldn’t be a problem. I could go through life happily ignorant of how to calculate an eigenvector, or even how to spell the word.

But in the olden days I was a college math and physics major, graduated with a 4.0 GPA, and was encouraged by my professors to consider going to graduate school. (I ultimately decided against it.) Many years later I did a bunch of data analyses in R as part of my blogging hobby and wanted to learn linear algebra (the area of mathematics that includes eigenvectors) to help me understand more advanced data science topics. I worked my way through a few chapters of an old edition of a linear algebra textbook, doing all the exercises (and blogging my solving them), but after almost nine years of working off and on I ran out of gas before getting to the chapter that covered eigenvectors.

And there things sat, until I stumbled across Math Academy, a new online education service (still in beta), via a mention on X. Doing a search for “Math Academy” on X and elsewhere brought up a bunch of enthusiastic testimonials, and since I was still peeved about not knowing about eigenvectors I was motivated to look into it.

As it happens, there’s a lot of online material explaining the Math Academy system, including an entire draft book, The Math Academy Way: Using the Power of Science to Supercharge Student Learning. So, I read the book, took copious notes on it, and read Math Academy material both informal and formal. After doing all that, I decided to spring for a subscription—which at $49 a month was not exactly a impulse purchase for me—and started out again on my journey to learn what an eigenvector is.

(Since I’m getting on in years, I’d like to travel fairly quickly. My goal is to complete the Math Academy Linear Algebra course and any prerequisites to it by the end of 2025. Then I’ll begin a second journey, with a goal of finishing the Probability and Statistics course and its prerequisite, the Multivariable Calculus course, by the end of 2026.)

Because Math Academy makes some significant claims about its service’s effectiveness, and others make even more extravagant claims on its behalf, I thought it was worth blogging about. Because there’s a lot of information out there, and I had a lot of thoughts myself, I divided it up into multiple posts, one per day. For reference, here’s the complete list:

• Part 1: My eigenvector embarassment. This post.
• Part 2: A skeptical prospect. My attempt to ignore the hype and start off with a pessimistic view of how effective Math Academy might be.
• Parts 2-8: These posts contain my notes summarizing the material in The Math Academy Way, along with my occasional comments. Feel free to skip these if you’ve read the book or aren’t interested in my comments on it.
  • Part 3: The sales pitch. How Math Academy (the company) pitches Math Academy (the product).
  • Part 4: Addressing objections. Why objections to other online learning systems (supposedly) don’t apply to Math Academy.
  • Part 5: Product features. The various learning strategies embodied in the Math Academy system.
  • Part 6: Customer responsibilities. What Math Academy users (and those who may be responsible for them) need to do to ensure they actually learn something.
  • Part 7: Technology brief. The nerd section, with in-depth explanations of how the Math Academy system works.
  • Part 8: Follow-up questions and notes. The Math Academy FAQ, and material that hasn’t yet made it into the main body of the book The Math Academy Way.
• Part 9: Customer feedback (non-pedagogical). My comments on my own experience with the Math Academy system, starting with the user interface and other aspects unrelated to the actual pedagogy.
• Part 10: Customer feedback (pedagogical). My thoughts on learning mathematics with the Math Academy system, based on using it for about a month and completing one class.
• Part 11: Final thoughts. Is Math Academy worth the money? Did I learn anything? Will Math Academy revolutionize mathematics education for everybody? Or even for just a few? And is that even worth worrying about in the Age of LLMs?

UPDATE 2026-01-15: After this series of posts I also published additional updates on my Math Academy experience:

• Update 1. I completed Mathematical Foundations III.
• Update 2. I completed Mathematics for Machine Learning.
• Update 3. I completed Linear Algebra.
• Update 4. I completed Calculus I.
• My one-year anniversary of using Math Academy.
• Update 5. I completed Calculus II.

In my next post I’ll look at Math Academy with a jaundiced eye.