In part 1 of this series I explained how my embarrassment at not knowing about eigenvectors led me to research and then sign up for Math Academy. I plan in this series to provide a lot of detail about both the Math Academy system and my experience with it—an experience that has been almost totally positive.

But before I do that, let me take a few moments to throw some cold water on Math Academy. Why I am doing this? I’ve spent almost my entire career working in sales groups selling software and hardware to large enterprises. Based on my experience, it’s best for both customer and vendor if a prospect starts the “sales journey” with a healthy amount of skepticism.

It’s too easy to let yourself be overly-dazzled by the promises of new “gee-whiz” technologies, or to be so suffering from painful problems that you leap to adopt the first product that comes along and promises to relieve that pain. That in turn can produce unhappy customers and a vendor distracted by trying to mollify them.

I tried to apply that way of thinking to my own case, doing research and looking for reviews of Math Academy (especially negative ones) rather than instantly reaching for my credit card and signing up for the service as soon as I read about it. So, please allow me to role-play as a skeptical and pessimistic prospective customer of Math Academy, at least for this post. Let’s start:

If you were a betting person and you were coming into this cold, you would bet that Math Academy is not going to “revolutionize education,” is not a “game changer,” and certainly will not have “civilizational impact”—to quote just a few of the things people have written about it.

Why is that? First, your prior position should be to hold as true the “null hypothesis in education,” as the economist Arnold Kling refers to it: that no education intervention—including online education technologies like the Math Academy system—will meet all of the following four tests:

  • It is experimentally validated as making a real difference (not just due to selection effects).
  • It is persistent (learning does not fade with time).
  • It is replicable by groups other than the original educators.
  • It is scalable to be able to support large numbers of students.

(The issue of selection effects is particularly relevant to Math Academy, since it appears to be attracting a customer base that is relatively more affluent, knowledgeable about mathematics, and motivated to learn than the student population as a whole.)

Audrey Watters, a critic of education technology, has her own version of Kling’s “null hypothesis”:

A friendly reminder that a meta-analysis of one hundred years of research on ed-tech looks something like this: some students showed some improvement on a standardized test in a specific subject area, after using ed-tech in a class taught by a supportive educator well-trained in that subject area and in the technology in question.

In other words, that’s about all that we can expect of the Math Academy system as well, a system whose roots lie in a context very much like this (a special mathematics program in the Pasadena, California, public school system).

Second, Math Academy touts itself as being based on the “two sigma problem” (or, more optimistically, “two sigma solution”) work promoted by education researcher Benjamin Bloom: that certain types of education practice can elevate a student from the 50th percentile in rank to the 98th percentile, a jump of about two standard deviations (hence the “two sigma”). But apparently other researchers have not been able to fully replicate Bloom’s results, and reading José Luis Ricón’s discussion of “two sigma” research, including some of the techniques touted by Math Academy, left me fairly lukewarm about the possibility of true two sigma solutions, or even one or one-half sigma solutions.

Next, Math Academy the company is yet another in a long line of for-profit and non-profit organizations commercializing online education technologies enthusiastically hyped as revolutionizing education. Your a priori assumption should be that it will fail to live up to that hype, just like its predecessors. Those failures have been exhaustively documented by Audrey Watters; see for example her comments on “The 100 Worst Ed-Tech Debacles of the Decade” (that decade being the 2010s).

Finally, there are various reasons why Math Academy is unlikely to have much impact on education as a whole, at least in the US, where the public school and higher education systems are characterized by a set of entrenched practices and often misaligned incentives. Many of these reasons are discussed in The Math Academy Way, and I’ll cover them in due time. However, they clearly limit the market that Math Academy can address, at least in the near term.

Where then might Math Academy find some traction? There are three obvious markets:

  • Homeschooled students, whose parents and guardians have rejected the public school system and are thus not restricted by public school ways of teaching.
  • Mathematically talented students (high 90s percentiles), who again are not often finding what they need in the traditional education system.
  • Adult learners (like me) who are looking to refresh and recover their mathematical knowledge or learn new areas of mathematics, whether in support of career goals or simply as a hobby.

These are not trivial markets. For example, the National Home Education Research Institute claims there were over 3 million US home-schooled students in 2021-2022. There are over 100,000 US students taking AP Calculus BC (equivalent to a university calculus course). There are also millions of people who subscribe to math-centric video channels like 3Blue1Brown.

However, the cost of Math Academy and the level of student motivation required to complete its courses will likely combine to limit its appeal among those three groups. There may be enough takers to support a sustainable business—50,000 customers at $500-600 a year would be a $25-30 million revenue opportunity—but it’s not clear that there’s enough of a market to achieve the broader ambitions of Math Academy’s founders and staff, much less to have “civilizational impact.”

So, how might Math Academy fail? Let me count the ways:

  • The system could fail to meet its education goals, such that students going through the program don’t show much actual improvement in test scores and grades.
  • The company could fail to meet its financial goals, not attracting enough customers to keep it solvent and then going out of business.
  • The company could be taken over by a larger company that proceeds to “enshittify” the service: for example, turning it into a free service with ads to attract more customers, and then optimizing for “engagement” (and consequent ad viewing) rather than learning.

Math Academy is apparently both self-funded and profitable, and its founders plan to keep it that way. But “everybody has a plan until they get punched in the mouth,” and we can’t rule out the possibility that Math Academy will get “punched in the mouth” financially or otherwise and will have to look to outside investors or even an acquirer for help.

However, the most likely scenario is that Math Academy will remain a profitable company, but will never break out of its initial niches of motivated and relatively affluent home-schooled students, mathematically-talented students, and math-interested adult learners. Most people would count that as a success in business terms, but as noted above I doubt it would satisfy the Math Academy founders and staff, who presumably want to change the entire field of mathematics education for the better.

But those concerns are for the future, which remains unwritten. In the meantime, I still don’t know what an eigenvector is and would very much like to. So, on to part 3, in which I consider the Math Academy sales pitch as embodied in the book The Math Academy Way.