CEGEP: It's More than a JuCo

Increasing inequity and the article The College Try brought this post (drafted long ago) back to my mind.

On his Disparate blog, Alexandre Enkerli wrote Defending Quebec's CEGEP System, which includes a somewhat lengthy description of the system.

What is a Cegep?

... A Cegep is a post-secondary institution («Collège») which serves both as a comprehensive («Général») transitional period between secondary school and university as well as vocational («Professionnel») training («Enseignement») in fields like nursing, robotics, or computer science. People in the U.S. could think of it as a blend of a vocational school, a community college, a prep school, a continuing education program, and a two-year liberal arts college.

My favorite thing about the Cegeps is that they are all those things. They bring together future literature majors, electricians, and computer geeks. In the U.S., post secondary schools are heavily segregated. Once everyone graduates from high school (which has its own magnet schools and schools within schools), the real segregation sets in. Future nurses and future computer programmers are rarely in classes with people planning to work a trade. This fractious nature of our educational system helps feed our fractured society.

Cegeps get all these people in the same place with many of the same core classes. Cegeps are also centers for continuing education, so many students are non-traditional students who are looking to change careers or simply have a thirst for learning. The cross-pollination of ideas and perspectives is a really great idea.

Cegeps are also cheap. This allows many students to "find themselves" professionally and academically. Having a place in which adolescents can transition from secondary students into independent people is important. This allows them to make mistakes, take chances, and have more confidence and competence once they move on to the next stage (whether that be a vocational school or a university). It's a bit amazing that we have so many teenagers making incredibly important and expensive decisions. Many people know someone who spent over $100k for a university education (perhaps not even resulting in a degree) that turned out to be useless for them. How much better would it have been to go someplace to figure it out without the stigma of not heading off to a university?

Constant Assessment: Making Data Useful

Assessment isn't about assigning a number to determine how a kid can perform on a particular day. Assessment is about figuring out what a kid does and doesn't understand (so far).

Actually, lots of assessment is about that first thing. Educators spend lots of time, energy, and money assigning numbers to kids. What about this: We shouldn't spend most of our assessment energy on the pedagogical equivalent of autopsies. We should spend more time developing, performing, and using formative assessments that help teachers teach and students learn.

In an earlier post, I said that we should assess more and test less. Here are a couple folks who have some ideas that seem related to this idea.

Bernard Bull (on Twitter @bdean1000) has Educational Publishers & Content Providers: The Future Is About Analytics, Feedback & Assessment, which puts the idea of a constant stream of assessment data into the proper context. It isn't about testing. It isn't about bypassing teachers. It's about using data to help students learn and teachers teach more effectively.

"... each action individually and collectively becomes a new data point that can be mined and analyzed for important insights."

His diagram showing how interactive content could be part of an effective part of a learning system is a call to action.

There's also Kristen DiCerbo's (on Twitter @kristendicerbo) work including these articles: How More Data Helps Us, Why an Assessment Renaissance Means Fewer Tests, All Fun & Games? Understanding Learner Outcomes Through Educational Games, and many others. I found Dr. DiCerbo's research when I searched Google for "invisible assessment," which is the same as what I have called "sneaky assessment." Whether it's "sneaky" or "invisible," this sort of assessment is where we should be headed.

eLearning content developers (such as myself) need to do a better job of making this happen, and teachers need to a) demand data-producing interactive content, and b) commit to really using it to help students learn more effectively.

Assess More; Test Less

Think about driving from Paris through pastoral northern France to Arromanches in Normandy.

If someone asked you how the drive was, would you simply report your successful, on-time arrival? Probably not. You'd probably mention the lovely scenery or how the trip made you feel or some other salient detail from the trip.

While you're on the drive, would you only care about your car's speed? Probably not. You'd also care about various aspects of your car's health as well (e.g., fuel level, engine temp, and RPM). If you don't pay attention to various aspects of your car's health, it could leave you stranded by the side of the road.

For years, online curriculum developers have focused on objective-driven courses. We measure how much content students have mastered, and we keep them aimed at consuming more content so they can master more content. We've been focused on the destination.

We need to spend more time thinking about the journey, and to do that, we need to make better use of modern technology and data. We need to stop testing so much, and start assessing more. Much more. Actually, we need to assess all the time. Let's group the kinds of things we should assess into two general categories:

Type 1 States: Content Knowledge and Readiness
These assessment are tied to specific objectives with a goal of mastery. We do a decent job of assessing content knowledge, but I don't think that many people do a robust enough job of assessing readiness. With tools like knowledge spaces, we should be able to figure out exactly what skills and knowledge a student needs to have in order to be ready to learn any topic. Why not figure that out ahead of time?

Type 2 States: Metacognition, Motivation, and Perseverance
As a student goes through any lesson, we should find ways to measure how they are reacting to the lesson. These three aspects go up and down through the course of a lesson, unit, and course. You don't master them any more than you master the fuel in your car's tank, but they are critical to engaging with content and truly mastering any field. Games and problem-based learning can be powerful tools for helping students learn skills, but perhaps they can be even more helpful when developing and measuring students' motivation and perseverance.

We need to measure Type 1 States frequently via truly formative assessment.

We need to measure Type 2 States continuously through every activity of a lesson.

Once we can measure both types of states well, we'll be able to help students learn and teachers teach more effectively.

Training: Measuring Effectiveness with Kirkpatrick's Levels

I have been a devotee of Kirkpatrick's 4 levels (he called them steps, but they are often called levels) of assessing the effectiveness of training since I read about them in the late 90s. I've used Kirkpatrick's levels as a framework for training evaluation systems for several projects at various companies.

Don Clark's Performance Juxtaposition site has Kirkpatrick's Four Level Evaluation Model, which describes the model quite well. Kirkpatrick's original article can be read here and Kirkpatrick Partners (the official site) summarizes the levels this way (my comments about how each level is assessed in parens):

• Level 1: Reaction - To what degree participants react favorably to the training (smile sheets)
• Level 2: Learning - To what degree participants acquire the intended knowledge, skills, attitudes, confidence and commitment based on their participation in a training event (end-of-course test/project)
• Level 3: Behavior - To what degree participants apply what they learned during training when they are back on the job (follow-up survey/interviews about how trainee transferred training to the job)
• Level 4: Results - To what degree targeted outcomes occur as a result of the                     training event and subsequent reinforcement (follow-up analysis to determine how trainee's organization benefited from trainee being trained)

We trainers like to see all top marks on our smile sheets (the surveys that assess Level 1), but most of us realize that only means the attendees liked us enough to not hurt our feelings. High scores on smile sheets can indicate good food, long breaks, or a great sense of humor.

Trainers also like to see students actually learn something. I used to do training for systems integrators and programmers, so I really liked to see them actually create a working web app. This allowed me to assess what they learned (Level 2). High scores on an end-of-training assessment can indicate either an easy test or that learning happened, but even in the latter case do not guarantee that learning helped trainees in their jobs.

What we need to be able to assess is how well trainees are able to transfer knowledge from training to their jobs (Level 3), and ultimately how that translates into better results for their organization (Level 4). Following up with trainees and their supervisors isn't easy, but every organization should commit to it for critical training efforts. This sort of analysis benefits the organization receiving the training (did we get what we needed?) as well as the training organization (are we really being effective?).

BTW: Clark's Big Dog, Little Dog blog and his twitter feed @iOPT are worthwhile reads for anyone in the training field.

What an Instructional Designer Can Learn from StudyBass

My wife bought me an electric bass for xmas. I have never played a musical instrument, so this represents a big new challenge.

When I wanted to figure out some of the basics of how to play, I wandered around checking out YouTube videos and sites with music to play, but eventually I found StudyBass.com. This works, and I think it's because it reflects pedagogy I have used and valued for years.

To be honest, many teachers and content developers could learn from StudyBass. Here is the StudyBass instructional model in a nutshell:

1. Theory is presented, but is connected to skills and to well-known songs from various genres so you can do what the theory says and hear how it applies to familiar songs. Hearing the roots and fifths pattern in Under Pressure by Queen is a pretty powerful way to verify how the theory and practice can lead to something great.
2. Each video of how to perform a skill is an animation that shows the finger positions very clearly. This is more helpful than watching someone do it in a live-action video because human instructors usually go too quickly and it's hard to see exactly what they are doing (their hands get in the way of seeing what their fingers are doing).
3. Each practice exercise is supported by various modes. You can a) hear it, b) read it in standard sheet music notation, c) read it in bass tab (which is very simple), and d) see it in alpha tab (which is like a bridge between the over-simplified bass tab and the pure sheet music). All these different modes allow for multiple learning styles, but also allow a student to gradually decrease the scaffolding support. You can start with the alpha tab (using bass tab as a fall-back option when you're stuck), then move to only the sheet music or only the audio.
4. Exercises progress in a logical order for each lesson. Just about every lesson has exercises, and they are great.

The only thing I would do differently is to start each lesson with a YouTube video of a great song that makes use of the upcoming concept. Motivating with the goal can be powerful.

What StudyBass has helped me appreciate even more than before:

• Connecting theory, practice, and real-world applications is a great way to motivate and build deep understanding.
• Students benefit from guided practice that includes scaffolding they can gradually reduce.
• When it comes to visuals, sometimes less is more.
• Small chunks of instruction can lead to larger chunks of guided practice and then unguided practice.

Now to get back to learning chord roots and how they connect to A Hard Days Night by The Beatles.

Angela Lee Duckworth: It's all About Grit

Reading articles about the misuse of the SAT reminded me of Angela Lee Duckworth's TED talk titled The Key to Success? Grit. 

Some of my strongest performers did not have stratospheric I.Q. scores. Some of my smartest kids weren't doing so well.

Dr. Duckworth studied grit at West Point, in K-12 schools, at spelling bees, and at private companies. The main predictor of success in these diverse contexts was grit. There are articles all over the place about Dr. Duckworth (a MacArthur Fellow); here is a link to an article at National Geographic: Grit Trumps Talent and IQ. One of her key points is that we need to figure out how to measure and instill grit. We understand how to measure IQ, but we don't understand how to measure grit, which is almost certainly more important.

I hope Dr. Duckworth and others like her figure out some good ways to measure and develop grit. We need to learn how to help every student be more gritty.

The Ivy Testocracy and the Unfortunate Misuse of the SAT

Salon brings us an article by Lani Guinier Ivy League’s meritocracy lie: How Harvard and Yale cook the books for the 1 percent, which is an excerpt from her book The Tyranny of the Meritocracy: Democratizing Higher Education in America.

First of all, let me be clear: I have nothing against elite schools, but ....

Ahh, for the days when elite schools were filled with people who inherited their privilege. Back then, most students at elite schools knew they were fortunate to have been born into "good" families, while those who didn't get in could rest easy that they were simply less fortunate.

Now, those at elite schools believe they deserve their good fortune while those who don't get in think they are less deserving. It's really too bad on both sides. I'm not saying that people at elite schools aren't smart. I'm simply agreeing with Dr. Guinier that a student's socioeconomic status is an excellent predictor of their aptitude as measured by the SAT.

Guinier's article reminded me of something I read in Atlantic Quarterly back in the mid-90's, Nicholas Lemann's The Great Sorting. One quote from Lemann's article stands out to me:

"Broad-scale testing in America was intended to be two things at once: a system for selecting an elite and a way of providing universal opportunity.... An irony of the American meritocracy, now that it has been in operation long enough to produce not just future leaders but present ones, is that the leaders chosen by a mechanism designed to be perfectly open and fair are widely regarded as a pampered, out-of-touch, undemocratic in-group...."

Also, SAT scores are not a particularly good predictor of later success. As Guinier says,

"... college admissions officers at elite universities today ... when asked what predicts life success—[say] that, above a minimum level of competence, “initiative” or “hunger” are the best predictors."

Organizations such as schools and businesses are not looking for people with high IQs or great SAT scores. What they want are people who are driven. People with grit. The problem is that we don't know how to measure "grittiness" well. We're great at measuring IQ and "aptitude," but those traits are much less helpful.

Much of this criticism of the SAT is mirrored by Todd Balf's article in the New York Times Magazine, The Story Behind the SAT Overhaul, which uses much of the same information to explain why David Coleman is overhauling the SAT. I hope Coleman's work is effective at changing the dynamic between the SAT, colleges, and students of every stripe. We need to move away from this testocracy.

Blogs Worth Reading: Thought Leaders

Here are some more blogs worth reading. These folks are curriculum people, teacher educators, and other thought leaders in the K-12 education space who are former K-12 teachers. Each is now out of the K-12 classroom, but focused on helping teachers teach and students learn.

Dan Meyer (@ddmeyer)
Blog: dy/dan
Subtitle: less helpful

Matt Townsley (@mctownsley)
Blog: MeTa Musings
Subtitle: ...where Math education, Technology and Assessment meet

Jason Buell (@jybuell)
Blog: Always Formative
Subtitle: Assessment is a conversation.

Dean Shareski (@shareski)
Blog: Ideas and Thoughts
Subtitle: Learning Stuff Since 1964

I could go on and on, but I figure I'd keep this list short (for now). Note that I post these links without commentary. You should check them out for yourself. If you know others I should add to my list of must-reads, please send them along.

Blogs Worth Reading: Math Teachers

I have found a group of math teachers who have made a really strong online community. Together, they are questioning, innovating, and making positive change happen and then sharing through blogs and tweets. It's incredibly inspiring.

Honestly, this is the best use of blogs and Twitter I have seen yet. My favorite thing about all of them is that they clearly have a growth mindset when it comes to their craft. They are growing as teachers in a collaborative, sharing way.

Anyone who is a new math or science teacher should read these blogs and follow these folks on Twitter.

That said, here is a list of some of my favorite math teacher bloggers/twitterers.

Kate Nowak (@k8nowak)
Blog: f(t)
Not only does Kate solicit ideas from the twitterverse; she actually puts the ideas into action and shares the results (regardless of how successful they were).

Sam Shah (@samjshah)
Blog: Continuous Everywhere but Differentiable Nowhere
Sam's post on the Blogotwitterversphere is great.

Fawn Nguyen (@fawnpnguyen)
Blog: Finding Ways
Great sharing, especially with the 3Acts.

Geoff Krall (@emergentmath)
Blog: emergent math
A nice mix of real-world application-driven stuff as well as insightful commentary.

Check these people out. I find them all to be informative and inspirational. I know there are others who are really good (my list of a few thought leaders who are not in the classroom will follow in a later post), but these are quite good. If you know of other great ones, please share.

Zoran Popovic: He's Got Games

This is the first in an occasional series on researchers I think are doing interesting work. Some will be well-known, but others might be somewhat less so. Regardless, I will share some thoughts on why more people should be putting this person's research into action. These aren't researchers who would only be interesting to someone working in an ivory tower. Each one does interesting work that you could use.

Dr. Zoran Popovic does a lot of work, but one thing he does really well is make games. As a matter of fact, he worked on algorithms that are used in the video game Destiny. He also cares quite a bit about how to help students. Notably, how to use hints to help students learn.

A Few of His Projects

Dr. Popovic runs the Center for Game Science at the University of Washington. CGS has done a bunch of good stuff including:

• FoldIt provides a context in which people with no knowledge of biochemistry can solve complex protein structure problems that help scientists solve real problems. It's actually pretty darn remarkable.
• Refraction is an interesting (and effective) way to learn fraction concepts and operations.
• DragonBox allows (even very young) students to learn algebra.

Popovic also runs a non-profit called EnLearn, which has built an adaptive learning platform. The EnLearn platform is not designed to replace teachers. It's designed to help them be more effective.

My Favorite Popovic Idea

So much to choose from, but I'm going geeky. I really like his Trace-based Framework for Analyzing and Synthesizing Educational Progressions. This reminds me of the Knowledge Spaces idea that lies behind ALEKS. It's computational and theoretical, but when you see it described well, it makes total sense. To a great extent, these traces are a big part of why EnLearn is so powerful.

Many people are making educational games and some people have built adaptive learning platforms, but very few people are doing either as well as Dr. Popovic and his teams.

Math Illiteracy

On his Uncertain Principles blog, Chad Orzel provides an interesting look at the problem of our culture's math illiteracy in The Innumeracy of Intellectuals. Professor Orzel's essential point is that people are generally ashamed of a lack of knowledge about art or culture, but can have a sense of pride about their lack of mathematical knowledge.

I don't think that everyone needs to love math any more than everyone needs to be an artist or a musician. On the other hand, here are three reasons I think everyone should master some key mathematical skills and concepts:

1. Math matters: We are constantly bombarded by information and much of it requires some analysis. Sometimes, we need a solid grasp of logic, and at other times, we need to understand statistics or probability. Math helps us make sense of many situations.
2. Math is helpful: The more math you know, the more ways you can see that it can be helpful as a way to solve real-world problems.
3. Math is beautiful.

Point 1's implications: I think everyone needs some knowledge of algebra, probability, and statistics. That's it. No calculus. No trigonometry. Nothing too fancy. If you can't make sense of the data and statistics that are part of modern life, then you can't make good medical, financial, or political decisions.

Points 2's implications: Teachers need to find ways to get students using math that matters and doing math that is just better. Dan Meyer has a post Real Work v. Real World. Doing crappy work in a real-world context is probably worse than doing interesting work in a made-up context. Math students need to do more good work.

Point 3's implication: It just is. If you haven't seen beauty in math, then you are hanging out with the wrong math teachers (everyone hangs out with a bunch of math teachers, right?).

Anyway, math matters, it's helpful, and it's beautiful. We need to use these truths to draw more people in and keep them engaged with learning math.

Driven by Data: Paul Bambrick-Santoyo's Model

In his books Driven by Data and Leverage Leadership, Paul Bambrick-Santoyo describes a model for facilitating effective teaching and using a focus on data-driven instruction to build effective schools.

Bambrick-Santoyo has a remarkable track record of turning around poor schools. His strategies don't require any fancy technology (though I think technology could help), but they do require that teachers and administrators break from some old ways of thinking.

At its core, Data-Driven Instruction (DDI) relies on believing that assessments are worthy goals. Many educators who hate these tests spit out the phrase "teaching to the test" as if it were an epithet. The negative attitude towards standardized tests is understandable. For many schools, end-of-year assessments are painful autopsies that expose their students' and teachers' deficiencies when measured by a meter stick that has little to do with what went on all year.

It doesn't have to be this way.

1) State standards for K-12 are getting better. I don't want to wade too far into the Common Core State Standards (CCSS) battles, but what I think many educators can agree on is that most states have created better standards over the past couple decades. Coherent curricula matter, and CCSS has made people pay attention to this. The standards are not a curriculum, but when standards are focused and well-organized, they can help schools develop coherent curricula.

2) More states are adopting end-of-year assessments that don't suck. This is critical. As states adopt higher-quality assessments to measure how students do against their better standards, teaching to the test isn't such a bad thing. Teaching to the sort of assessment I took in school (nothing but multiple choice questions with low cognitive demand) would be criminal. As assessments become much more sophisticated and include constructed response, technology-enhanced items, and even performance tasks, teaching to these new assessments shouldn't be so repugnant. According to a RAND study, these new assessments can lead to better instruction, but only if (among other things):

• they are part of an integrated assessment system that includes formative and summative components and
• the new assessments are a component of a broader systemic reform effort.

3) If the standards are better and the tests are better, then having teachers use benchmark tests (Bambrick-Santoyo calls them Interim Assessments) to see where their students are can be incredibly powerful. This isn't about having every school day be all about assessment. This is about mapping out a plan for how your students will get to the finish line and then checking at a few points along the way to see how you (the teacher) and the students are doing. This isn't about giving each student a number. It's about understanding what they do and don't know and what the teacher needs to do differently to help them make progress.

It's a self-fulfilling prophecy either way: Teachers who fight the test will have students who struggle on the autopsy. Teachers and schools who find ways to improve the instructional processes with the assessments in mind have a better chance of reaping great rewards for their students.

Education and Business: A Tough Combo

Achival's Blog brings us Why Education Startups Don't Succeed. It's an interesting read, especially for anyone who has been involved in the online education business over the past decade or so. The entire piece is worthwhile, but I particularly like the list at the end. The first bullet is pretty sobering:

Don’t believe that building a better product will make you successful. Delivering something for cheaper will. Even if that cheaper thing is lower quality. This is usually repugnant to most well-educated entrepreneurs.

Ugh. Yeah, that is a bit repugnant. I want to believe that a better product will succeed, but I know that having a great product is neither necessary nor sufficient. I know this because I have seen some companies get paid for pretty poor products while companies with brilliant people and products languish. As long as a product is cheap and has the perception of filling a need (thanks to good marketing copy and/or a well-spoken champion), it has a chance to succeed.

Am I a little bitter when I see someone cash in on a crappy product? Yes, I am. I respect their chutzpah, but am disappointed in a system that can't identify, use, and reward quality.

TechCrunch says that Education Technology Startups Raised Over Half a Billion Dollars in Q1 of 2014. People are still trying to build better educational mousetraps. I respect the effort and hope the good ones succeed.

Over the last couple decades, I have worked for a handful of small companies trying to change education and/or training. Some have seen some success, while others faded away or imploded. I am quixotic enough to keep trying, but realistic enough to know that developing some amazing solution won't guarantee success.

The Wrath of Khan

With his tutorial videos, Khan Academy has made a big splash in the education world. Having Bill Gates sing your praises in a TED Talk is a sure way to get a lot of attention.

Over at Action-Reaction, Frank Noschese has an article Khan Academy: My Final Remarks. I agree with many of his points. Here is a quote from near the end of the post:

Khan Academy is just one tool in a teacher’s arsenal. (If it’s the only tool, that is a HUGE problem.) Khan Academy can be useful for some kids as a vehicle (build skills) to help them get to better places (solving complex problems).

Here are some other bloggers putting KA into some reasonable context:

• On his Quantum Progress blog, John Burk also chimed in with Project Euler vs Khan Academy: The Future of Online Learning.
• Mathalicious's blog has Socrates & C-3PO. 
• On his MAA blog Devlin's Angle, Professor Keith Devlin brings us The Difference Between Teaching and Instruction. 

It's comforting to see that I am not alone in thinking that if Khan is really the future of education, then we are in deep doodoo. Frankly, I would not really call Khan Academy’s stuff “education,” but rather would consider it “instruction.” If we turn education into nothing but a series of activities that a microchip can perform, then we are on a very dangerous path.

I don't mean to be a Khan basher. When using KA as a way to support struggling students, or as a way to help flip a classroom and allow teachers to focus on engaging with students, it can have great value. The Daily Riff has The Flipped Class Manifest, which provides some insights into how and when flipping a classroom can work, as well as a nice set of links to articles that provide more depth.

Back in 2007 (before I ever heard of Khan Academy), I started developing instructional videos that are somewhat similar to Khan’s work. I think they are great tools, but they are not the be-all and end-all of online education. We need to continue to innovate, but shouldn't get too carried away by the latest educational fad and think it will solve all our problems.

The Changing Nature of Being Educated

I've been reading "The Information: A History, a Theory, a Flood" by James Gleick, and in it, he has a quote that I had to dig a little deeper on. Here is a slightly longer version of something Gleick quoted:

For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in [this technology] will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom,....

So, who was this Luddite railing against technology? Though this could easily be my mother or father, it was actually Socrates in Plato's Phaedrus. He was warning of the perils of written language, just as others would eventually complain about the printing press and then radio, film, TV, and every other transformative information technology.

In this day and age, what does it mean to be educated, and how has that definition changed over time? In Plato's time, you could be considered educated even if you were illiterate. In the 1970's you needed to know how to read, but could still be considered well-educated if you had no clue about how to use a computer. What about now? More importantly, what about 10, 20, or 30 years from now?

Note: I am not trying to wade out into the depths of the battles about whether technology is making kids worse students. New York Times has Technology Changing How Students Learn, Teachers Say, which does wade out into those scary waters. I just think the changing nature of what it means to be educated is an interesting thing to keep an eye on.