Why do universities fail so often?
Faculty reward systems
Professors get tenure and promotion by publishing journal articles. That’s also how they get social rewards, like respect from their peers.
Good teaching doesn’t count for promotion. Many professors think that time spent on teaching is wasted.
Their attitude isn’t surprising. There’s a famous article in business, On the folly of rewarding A while hoping for B. The title says it all.
Most professors do care about students, and enjoy working with them. However, most don’t try to improve learning. It’s not part of the reward system, or the culture.
Good teaching is hard to measure
A professor’s research productivity is (relatively) easy to measure. Count publications in peer reviewed journals. Count citations.
Good teaching is harder to measure. Do you look at course grades? That encourages grade inflation, and “dumbing down.” You could use student performance on standardized tests. There are standardized tests for some intro courses, but not for most courses.
Many campuses ask students to evaluate professors. In principle, students can accurately assess some things, like whether professors returned homework promptly. However, some things students can’t asses, like whether the professor chose the right material. How would students know?
A physics professor had an interesting experience with student evaluations. He gave students tasks that lead to better learning, as measured by a standardized test (intro physics has one). Even though the students learned more, they gave the professor worse evaluations.
Who gets hired for professor jobs? People good at research, or who have good potential. Most professors haven’t taken even one course in learning or teaching.
Who gets into Ph.D. programs? People who were good students in their disciplines (physics, computer science, whatever). Teaching doesn’t enter into it.
Learning a skill like programming is tough. It takes more than one course. A computer science student takes, say, ten programming courses.
Students learn the most when the first course feeds into the second course, which feeds into the third course, and so on. Some university departments coordinate well. Others do not. Professors teach in isolation. Students get a bit of this and a bit of that, rather than a coherent, complete set of skills.
Hate the game, not the player
In the U.S., this system evolved in the first part of the 20th century. Professors working today did not design this system themselves.
Poor teaching is not the result of lazy professors. The problem is the system that selected them, and shaped their behavior. That system does not value good teaching.