Have you met Webb's Depth of Knowledge in all its reformy goodness. I just spent a couple of blood pressure-elevating hours with it. Here's the scoop.
In Pennsylvania, our state department of education has Intermediate Units which are basically regional offices for the department. The IU's do some useful work, but they are also the mechanism by which the state pumps the Kool-Aid of the Week out into local districts.
Today my district hosted a pair of IU ladies today (IU reps are typically people who tried classroom teaching on for size and decided to move on to other things). As a courtesy, I'll refer to them as Bert and Ernie, because one was shorter are chirpier and the other has a taller frame and a lower voice. I've actually sat through DOK training before, but this was a bit clearer and direct (but not in a good way).
Why bother with DOK?
Bert and Ernie cleared this up right away. Here's what was written on one of the first slides in the presentation:
It's not fair to students if the first time they see a Depth of Knowledge 2 or 3 question is on a state test (PSSA or Keystone).
In other words, DOK is test prep.
Ernie showed us a pie chart breaking down the share of DOK 2 and 3 questions. She asked how we thought the state will assess DOK 4 questions? Someone went with the obvious "on the test" answer, and Ernie said no, that since DOK 4 questions take time, the Test "unfortunately" could not do that.
There was never any other reason. Bert and Ernie did not even attempt to pretend to make a case that attending to DOK would help students in life, aid their understand, or even improve their learning. This is test prep.
Where did it come from?
Webb (it's a person, not a piece of jargon) developed his DOK stuff in some sort of conjunction with CCSSO. Ernie read out what the initials stand for and then said without a trace of irony, as God is my witness, "They sound like real important people, so we should trust them." She did not mention their connection to the Common Core which, given the huge amount of CCSS love that was going to be thrown around, seems like an odd oversight. The presenters did show us a graphic reminding us that standards, curriculum, and assessments are tied together like the great circle of life. So there's that.
How does it work?
This turned out to be the Great White Whale of the morning. We watched two videos from the Teacher Channel that showed well-managed dog and pony shows in classrooms. Bert noted that she really liked how the students didn't react to or for the camera. You know how you get that? By having them spend lots of time in front of the cameras, say, rehearsing their stuff over and over.
The first grade class was pretty impressive, but it also only had ten children in it. One of my colleagues asked if the techniques can be used in classes with more than ten students (aka, classes in the real world) and that opened up an interesting side note. The duo noted that the key here is routine and expectations, and that you need to spend the first few weeks of school hammering in your classroom routines so that you could manage more work. One teacher in the crowd noted that this would be easier if all teachers had the same expectations (apparently we were all afraid to use the word "rules") and Ernie allowed as how having set expectations and routines from K through the upper grades would make all of this work much better. "Wouldn't it be lovely?" she said.
Because when you've got a system that doesn't work very well with real, live children, the solution is to regiment the children and put them in lockstep. If the system and the childron don't mesh well-- change the children.
Increasing rigor!
You might have thought this section would come with a definition of that illusive magical quality, but no. We still can't really explain what it is, but we know that we can increase rigor by ramping up content or task or both.
We had some examples, but that brought up another unsolved mystery of the day. "Explain where you live" (DOK 1) ramped its way up to "Explain why your city is better than these other cities" (DOK 3). One of my colleagues observed that this was not only a change in rigor, but a complete change of the task and content at hand. Bert hemmed and hawed and did that little I Will Talk To You Later But For Right Now Let's Agree To Ignore Your Point dance, and no answer ever appeared.
So if you are designing a lesson, "List the names of the planets" might be a DOK 1 question, but a good DOK 3 question for that same lesson might be "Compare and contrast Shakespeare's treatment of female characters in three of his tragedies."
Audience participation
Bert and Ernie lost most of the crowd pretty early on, and by the time we arrived at the audience participation portion (two hours later), the audience seemed to have largely checked out. This would have been an interesting time for them to demonstrate how to handle a class when your plan is bombing and your class is disengaged and checked out, but they went with Pretending Everything Is Going Swell.
The audience participation section highlighted just how squishy Depth of Knowledge is. Bert and Ernie consigned all vocabulary-related activities to Level 1, because "you know the definition or you don't." That's fairly representative of how test creators seem to think, but it is such a stunted version of language use, the mind reels. Yes, words have definitions. But there's a reason that centuries of poetry and song lyric that all basically mean, "I would like to have the sex with you," have impressed women far more than simply saying "I would like to have the sex with you."
There's a lot of this in DOK, a lot of just blithely saying, "Well, this is what was going on in the person's brain when they did this, so this is the level we'll assign this task."
DOK's big weakness
DOK is not total crap. There are some ideas in there that can lead to some useful thinking about thinking. And if you set it side by side with the venerable Bloom's, it can get your brain working in the same way that Bloom's used to.
But like all test prep activities, DOK does not set out to teach students any useful habits of mind. It is not intended to educate; it is intended to train students to respond to certain sorts of tasks in a particular manner. This is not about education and learning; this is about training and compliance. It's a useful window into the minds of the people who are writing test items for the Big Test, if you're concerned about your students' test scores. If you're interested in education, this may not be the best use of your morning.
"As a courtesy, I'll refer to them as Bert and Ernie, because one was shorter are chirpier and the other has a taller frame and a lower voice."
ReplyDeleteThe first two and a half paragraphs told me that Peter had had a sip of his secret awesomesauce just prior to writing this, but this quote about had me on the floor. But then, muppets are always unexpected. This would be one of the many reasons I'm a regular reader. Anyways, back to the (Lord help me, I almost said "text")...
If this happened in the private sector, the business would go under because of mismanagement.
ReplyDeleteThis DOK stuff is a hot mess, as my daughter would say. It certainly doesn't seem based on neuroscience. There's no clarity at all. You can't use the same stupid verbs for different "levels" if you want anyone to understand. There are no strategies to explain how to teach any of this. All the different things I've read about it put a lot of things at different "levels" since no one seems to understand it. (I like your Escher cognitive dissonance simile in your earlier article.) I disagree with a lot of the categorizations. For instance, I don't think knowing how to use punctuation is simple recall. My two daughters who successfully completed post-grad work still don't know how to use commas. I only punctuate as well as I do because I've read tons and have an eye for detail. The purpose of punctuation is clarity and that's what should be emphasized in teaching it. And different kids will have difficulty with different things. For example, I cannot write poetry or narratives. My brain doesn't work that way; it's totally analytical. I like Bloom's taxonomy better than DOK, though there are critics who say that any distinction between categories (levels) is artificial because any cognitive task entails a number of processes. And some say application should be before concept. That would fit better for people who naturally use inductive reasoning rather than deductive. Most of the DOK's "higher level" skills just seem to me like activities that they want you to be able to do on the tests. I'm going to continue this in a second post because I'm probably running out of room.
ReplyDeleteFirst to add to people having different aptitudes, I want to clarify that I totally appreciate literature, but in spite of reading tons of it, I still can't write any kind of narrative or poetry. Another different thing is that some people do better with concepts than terminology (like my son). But I also really have a problem with DOK throwing around abstract terms without defining them, starting with "deep knowledge." I would say deep knowledge is not easily seen or apparent; it's hidden out of sight because it's "underneath". It's the underlying principles and patterns that help you understand why and how everything works. (Like "deep structure" in sentence structure in linguistics.) To me these principles and patterns, the things that repeat and are always true, are the main things we need to be teaching. Webb seems to think deep structure means complexity. Then they're always saying that complexity is not the same as difficulty and complexity is what makes something "higher order" and "critical thinking". Difficulty means a task almost beyond one's ability, requiring skill, perseverance, patience, and guesswork and trial-and-error to solve. Complexity means having interrelated parts that are difficult to separate and understand. To me, as teachers, we're working with facts, concepts, and skills. Facts are details that are true (who, what). Concepts are the underlying principles (why). Skills are ease in the use of procedures (how). Facts need memorization; concepts need understanding; skills need practice. Application is knowing when and where to use particular concepts and procedures. Analysis is the procedure by which something is broken down into its component parts to see the relationships between them. The problem with DOK is that it's not clear what the underlying principles are, so how are they supposed to be understood?
ReplyDeleteI think some clarification is in order.
ReplyDeleteFirst, Depth of Knowledge dates back to the late 1990s from Norman Webb's involvement in helping states align their content standards to their statewide assessments.
Second, Depth of Knowledge was designed purely as an alignment tool, not as a classroom tool (or test prep). Along with Karin Hess, I was probably the first to drive hard for the classroom use of Depth of Knowledge back in 2007. In 2008, my company performed two huge studies of classroom rigor using Depth of Knowledge as one of the metrics. (The other metric was Bloom's Revised Taxonomy.) My Cognitive Rigor blog contains tons of articles on bare-knuckle uses of Depth of Knowledge in the classroom.
Third, professional development in Depth of Knowledge has been spotty because the Common Core rolled it out as a standards tool too fast. Professional development companies had a hard time getting the grasp of Depth of Knowledge, but had to roll out training materials quickly to remain competitive. So there are tons of subpar trainings out there; it looks like the blog author was a victim. Only a handful of trainers have any long-term experience with Depth of Knowledge. (Karin Hess is one of them; my company -- The Standards Company -- is another. Not sure who else.)
Fourth, Depth of Knowledge is not based on neuroscience, true. But that means little in terms of its credibility because it was not designed to describe neurological functions. Bloom's Revised Taxonomy has a closer tie to neuroscience, but that's because it more closely describes types of thinking. Depth of Knowledge describes the process of knowledge management. Six Sigma isn't based on neuroscience either, but it's still an industry standard.
As far as strategies to teach with Depth of Knowledge, may I refer to my own blog of 70+ articles on rigor? Most of the articles address Depth of Knowledge and its use in the classroom. http://cognitiverigor.blogspot.com/
Finally, I have been trying to get the Pennsylvania Department of Education to pull down its Depth of Knowledge chart, which makes a huge mistake of tying each level to action verbs. You can read about this chart here: http://bit.ly/1pleYyf
I appreciate your chiming in, John.
ReplyDeleteMy previous DOK training experience was with Jill Stine of Common Core Insititute and/or the Center for College and Career Readiness. I wrote about that here: "http://curmudgucation.blogspot.com/2014/01/today-state-of-pennsylvania-provided-me.html"
Interesting to see you describe DOK as a process of knowledge management. I think that may be the most useful short form description I've encountered. I will take a look at the materials you've linked.
Sadly, I don't have to go to PA DOE's website to see their crappy verb-based chart-- everybody in my building got to have their very own copy.
during our DoK "training" I wrote Doodoo on Krackers all over my handout. It seemed to help for awhile.
ReplyDeleteWhile I understand that the training may have failed to reach expectations, we are professionals and we should always try to learn as much as we can.
ReplyDeleteOver here in LA, we are also swimming in a sea of dok .... or rebranded bloom .... or costa 2.0. I know I have some verbs somewhere to make a floaty.
ReplyDelete