Details from a comprehensive report outlining the findings from a three-year assessment of the impacts of CCJ's Traveling Curriculum newsroom training program.
Engagement in workshop experience
Purpose and mission
Reflectiveness and critical thinking
Quality of news product
Use of tools related to particular elements of journalism
Communication
Innovations
Engagement in workshop experience
We expected that participation in the workshops would be viewed as a positive experience, since it provided an opportunity to revisit the fundamental purposes that draw most journalists to this line of work. Most newsrooms never break away from their daily routines long enough to reflect on the central issues in journalism, and mid-career training in this field has been available only sporadically to a chosen few. For this reason alone, serious efforts to bring training opportunities to newsrooms are likely to meet with widespread staff approval, and this is exactly what we found.
Consistently, in the vast majority of print and broadcast newsrooms that we visited, participants reported that their experience of engaging in the workshops was strongly positive. In post-session evaluations, when participants were asked to rate the overall workshop experience from 1 (outstanding) to 5 (poor), over 95% of respondents gave the workshops the highest ratings. This is an extraordinarily high proportion of high ratings. Even considering the inevitable “Hawthorn effect” referred to above, the 95% figure reflects an extremely positive overall response to the program.
Generally, participants responded that they found the workshops to be stimulating, informative, worthwhile, memorable, enjoyable, and thought-provoking. Typical is a comment of a reporter from a mid-sized newspaper: “Interesting and more useful than any other conference I have attended.” One journalist from a larger newsroom wrote: “The discussions of issues...were lively, engaging, and struck at the heart of what we are or need to be thinking about as we do our jobs.”
[top]
Purpose and mission
The most consistent theme of all the workshop modules was the importance of always keeping the public mission of journalism in mind; and all the workshops offered participants multiple occasions for reflecting on the most fundamental purposes of their work. For this reason, we expected marked impacts on participants’ awareness of the newsroom mission and their own journalistic purposes.
An analysis of interview data revealed strong and consistent effects on participants’ sense of journalistic mission and on their inclinations to care about the broader purposes of their work. These outcomes are connected to the Curriculum’s first goal, reinforcing the dedication of newsrooms and those who work in them to journalism’s traditional public mission. Accomplishing this mission requires an understanding of how journalism serves the public in ways that help them make informed choices about their lives and their roles as citizens in a democratic society.
71% of respondents, aggregated across all newsrooms, reported having a clearer sense of journalistic mission after the workshops. The print and broadcast newsrooms reported virtually the same levels of effect (71% of participants for print, 74% for broadcast), with only the Internet newsrooms reporting a somewhat lower effect (49%).
[top]
Reflectiveness and critical thinking
Another primary goal of the curriculum was to impart to participants the importance of critically examining their sources of information, the nature of their own biases, and, ultimately, the quality and validity of their news judgment. For this reason, we expected marked effects on the critical thinking capacities of participants. Critical thinking capacities are essential in participants’ abilities to make sound news judgments.
The interview data revealed clear effects on the willingness and ability of participants to use critical thinking in their journalism. The majority of respondents reported that they, or their newsrooms, had become more reflective about their work as a consequence of the workshops. They reported a greater tendency to apply critical thinking to their stories, in both the research and the writing phases. 58% of all respondents reported this change. Overall, aggregating responses for all varieties of newsrooms, 27% of participants said that bolstering their critical thinking and reflectiveness was the most valuable benefit of the workshops for them.
Print participants reported the critical thinking effect at higher levels than broadcast participants and at markedly higher levels than Internet participants: the numbers were, respectively, print 70%, broadcast 51%, and Internet 26%. Although broadcast newsrooms showed a bit less impact than print newsrooms in their critical thinking dispositions, they showed a strong effect in their capacities to make sound news judgments. 44% of broadcast participants reported making better news judgments subsequent to their workshop experiences. News judgment certainly includes the capacity for critical thinking, but it also refers to priority-setting in assigning, airing, and ordering stories, decisions about how much context to provide viewers, and a host of other choices regarding emphasis, proportional coverage, controversial content, and balance. Among broadcast participants, 64% said that the ability to make sounder news judgments was their most valuable takeaway from the workshops. Many participants from broadcast news attributed their improvements in news judgment to the “story stacking” exercise that played a central role in the broadcast training (see description of this exercise above). At every station, someone mentioned the particular value of this method, and over 40% of the broadcast participants spontaneously noted during their interviews that it was the most memorable part of their experience.
[top]
Impact on quality of news product
The most important consequence of clear purpose, critical thinking, sound news judgment, and high morale is the quality of the news stories that journalists and newsrooms actually produce. We expected to find improved quality in the actual reporting of the newsrooms that we visited. As noted above in the methods section, in order to assess quality of reporting, we combined two methodological strategies. First, we interviewed participants and managers about their perceptions of quality before and after the workshops. Second, we examined the actual “product” – print stories and broadcast rundowns – of the newsrooms before and after the CCJ visits. This kind of product analysis has rarely, if ever, been done in assessing the impact of journalism training programs. Results from both methodological approaches confirmed that the reporting product of participating newsrooms improved markedly after the CCJ visits.
1. Interview assessments of product quality
The most evident consequence of strong mission, good news judgment, and high morale is the quality of the news stories that journalists and newsrooms actually produce. The section on product analysis below reports direct assessments of newsroom products before and after the CCJ visits. The interview data, in contrast, offer indirect indicators, consisting of participants’ perceptions of changes in product quality due to the workshops. In all newsrooms combined, 31% of workshop participants reported improved quality in their newsroom’s news products. The effect was largest for medium-sized print newsrooms, where 52% of participants stated that the quality of their newsroom’s reporting had markedly improved due to the workshops. In those newsrooms, unlike in the larger ones, most or all staff members were able to attend the workshops; and, unlike in the smaller papers, there were enough staff to arrange multiple versions of small-group discussion sessions. Unlike in the broadcast training, the print newsrooms were offered full day-and-a-half workshops that were able to explore three modular journalistic topics in depth.
2. Direct assessments of product quality
The analyses of the product collections referred to above yielded a number of direct indications of improvements in the quality of news stories after the CCJ visits. For the print newsrooms, a sample of work done by individuals who participated in the workshop training and were designated by their newsroom managers as highly engaged in the workshop experience showed clear and consistent improvement trends. This designation did not necessarily set them apart from most other staff who participated in the training since, as noted above, over 95% of total participants in the CCJ workshops gave the workshops top scores on their overall ratings for value and interest.
Print product analysis
At eleven of the newspapers where we assessed work of individual reporters, the quality of work improved significantly subsequent to the CCJ workshops. Breaking down the quality assessments into sub-components, the work of the individual reporters whose work was analyzed showed improvements in the following areas: explaining the future impact and context of the news event; writing articles without inserting personal opinion; writing articles that gave the reader multiple perspectives on the conflict described by the story; and explaining in the story what is not known as well as what is known. For each of these quality components, no individual work declined in score from pre to post-test – all stories showed either positive change or none at all. Regarding the careful and limited use of anonymous sources, and the use of transparency in communicating story background to readers, there also was net improvement subsequent to the workshops. At three of the papers, on the other hand, participants’ work showed little or no improvement; and at two of the papers the assessed quality of work was scored lower by a small degree after the workshop experience. The work of individual reporters at one newsroom (not the same newsroom for the two instances) showed negative rather than positive change on the sub-components of quality noted above.
As could be expected, the print newsrooms taken as a whole showed no clear pre/post improvement trends, no doubt because of the inclusion in the product analysis of many stories written by reporters who had not attended the workshops. But overall, the analyses of individual reporter’s quality of work in the print newsrooms assessed yielded more direct indicators of pre/post improvement than has been reported in most prior evaluation studies.
We examined the work of two individual reporters as case illustrations of these general trends. For the sake of this report, we shall refer to these as PR-16 and PR-17. Both reporters work for mid-sized newspapers, one on the East Coast and one on the West Coast.
PR-16 is an enterprise reporter who covers a wide range of news and feature topics, including political, crime, environmental, and human interest stories. She is a talented writer who composes stories in an extremely engaging and literate manner. The stores that we collected from the periods prior to and after the CCJ visits all reflected her highly engaging writing style, with no noticeable change in this dimension subsequent to the visits. But on other product quality indexes, PR-16’s stories showed marked improvement after participating in the CCJ workshops. Although her stories predating the CCJ visits received relatively strong quality scores on most of the indexes, subsequent to the visits they were scored significantly higher.
For example, PR-16’s sourcing practices directly reflected the discussions from the accuracy and verification module that CCJ presented at her newspaper. After the CCJ visit, PR-16’s use of anonymous sources declined significantly. Prior to the visit, she used anonymous sources in 36% of her stories, a moderate amount; but after the visits, she used anonymous sources in only 10% of her stories, a very low figure. Moreover, her sourcing showed greater “transparency” – that is, she more clearly identified not only the source but the source’s relationship to the events in the stories and the source’s capacity to provide reliable information. In a post-visit crime story, for example, she offered full titles and backgrounds of every government source and noted that the defendant’s attorney, also quoted, was state-appointed. In a science story, she provided titles and backgrounds for all scientists quoted and also pointed out that they had authored a paper that had taken a position on the issue that they were commenting about.
In her interview with us during our assessment visit, PR-16 said, “I’m definitely more thoughtful about sourcing and verification issues.” She also said that she now had “More awareness of our [newspaper’s] credibility all around.” These reflections are consistent with the improvements noted above. Interestingly, she also said that one of her most valuable takeaways from the CCJ visit was learning about “making stories more positive, more entertaining, fun and easier to read, more relevant to readers’ lives.” In fact, as we noted, PR-16’s pre-visit stories were already so high on these dimensions that it was impossible to detect any post-visit improvement. It may be that reporters with special strengths frequently look for ways to build further on these strengths. When this is the case, training became more than a matter of filling in deficits in skill – it becomes an opportunity to move further toward excellence in craft.
Finally, PR-16’s post-visit stories showed modest increases in conveying the potential impact of stories on readers’ lives and discussing their implications to the broader community. Her stories also showed a small (and not statistically significant) increase in their tendencies to include a diverse group of subjects, including more frequent mentions of children, young adults, senior citizens, minorities, and people with multiple religious faiths.
PR-17 is a metro and general features reporter from a mid-sized Western newspaper. His pre-CCJ visit work reflected a straightforward, “no frills” approach to news writing. He generally used simple declarative sentences that conveyed the necessary information, but were not particularly colorful.
The quality of PR-17’s work showed significant post-visit improvement in the area of telling readers “what is not known” about the topic or event being covered in a story. This improvement hints at a greater effort to be transparent with readers, a key idea in the conscience and communication in the newsroom module to which this reporter was exposed. “I feel comfortable explaining myself and see value in it,” this respondent said during his assessment interview. This statement came in an interview response in which he made the case that his newsroom could do a better job of explaining how the editorial process works and how decisions are made. His improvement in relating what is not yet known to readers shows he found a tangible way to translate that feeling into his work.
This reporter also showed marked, although not statistically significant, improvements in using fewer anonymous sources, telling readers about the potential implications of topics covered in his stories, and presenting multiple viewpoints in stories in which there was conflict.
In general, PR-17’s writing was noticeably “snappier” and more interesting to read subsequent to his participation in the CCJ workshop as well. The pre-workshop sample of writing from PR-17 can be fairly characterized as rather formulaic, reliant on institutional sources, and lacking in creativity – especially in the story leads. Several of the stories failed to move beyond isolated instances to connect to broader themes and trends in the community, state, region, or country. They weren’t “poorly-written” stories by any stretch. But the writing in most stories was not particularly compelling, detailed, or deep; and the author seemed to stop short in places where the typical reader may have wanted or needed more information to judge why she should care about the story.
The most striking feature of PR-17’s pre-workshop product sample is his over-reliance on a specific single source for information in several stories. Six stories on four different topics contained quotes from the same local elected official.
PR-17’s post-workshop stories displayed more interesting leads, a wider diversity of sources, and efforts to present multiple viewpoints on the central conflict (where there was one) of stories. Additionally, the “nut graf” in many of these stories was more visible and more thoughtfully written.
“When I do in-depth stories in particular, I reference the training,” PR-17 said during an assessment interview one year after the CCJ workshop. “Especially on the concept of ‘stakeholders.’ Am I getting them? How does this story affect people? I try to do extra fact checking and get back to sources and make sure the tone [of the story] is appropriate.” This conscious effort to make stories more relevant and thorough, among other things, appears to have helped PR-17 improve the overall quality of his writing.
Broadcast product analysis
In the broadcast newsrooms assessed, the analyses found improvements in quality and treatment of first block stories. The average length of first block stories increased significantly – a noteworthy finding, since prior research has established a strong positive correlation between length of story segment and quality in broadcast news. In addition, subsequent to the CCJ workshops, there was a greater tendency for broadcast news stations to present stories – and, in particular, their first-block stories – in edited packages. This was noteworthy because edited packages reflect deeper and more contextualized treatments of information offered in the stories. Moreover, we found a greater tendency for broadcast stations to present campaign election stories in edited packages as well as serious lifestyle stories involving the lives and deaths of political leaders. In some of the newsrooms, less time was devoted to live crime reporting and more to live reporting of civic and political news.
Improvements in quality and treatment of first block stories were most pronounced in the broadcast stations that we examined for case examples, due to the seriousness with which these particular stations engaged in the training. In addition to the improvements noted above, we found in some of these stations an increase in live shots and packages in stories related to civic and political issues and a decrease in live shots and packages in stories related to crime. Both of these changes imply improvements in the quality and seriousness of the stations’ new coverage.
In general, our assessments of news product quality before and after CCJ training showed a marked impact on individual reporters in print newsrooms but not much effect on entire print newsrooms. In the broadcast training, in contrast, we found a number of positive effects at the whole-newsroom level. Part of this difference may be attributable to a methodological issue: the print product that we collected at the whole-newsroom level necessarily included many stories that were written by reporters who had not received the training; whereas in broadcast, where virtually everyone participated the training, the product that we collected almost always was done by people who had been trained.
In addition to this methodological artifact, it may be that whole-newsroom effects are easier to obtain in broadcast than in print (see the section “Impact on newsroom communications” below, where we report a similar whole-newsroom impact favoring broadcast over print). This difference might stem from a number of causes, including the more entrenched practices of print journalism, which evolved over many decades before broadcast news had been in existence. The collaborative nature of creating broadcast product could further magnify this difference.
One reporter, for example, might remember a fragment of a training lesson and refer to it while discussing with colleagues problems in reporting a complex story properly. This fragment can spark memories of the shared training experience among the photojournalist, the producer, the tape editor, and others who are working on the same story; and together they may retrieve enough of the training idea to solve the story problem. This scenario is more common in broadcast than in print, where reporters tend to work through story problems that they encounter without much consultation with colleagues.
BR-11 is a moderate sized Midwestern news station, and BR-3 is a strong broadcast station in a highly populated Southern community. Analysis of the newscasts aired by BR-3 and BR-11 before and after their CCJ workshops revealed that these stations had improved in several important areas after the CCJ workshop.
We coded these stations’ newscast rundowns for story treatment – a measure of the manner in which the story was produced and appeared to viewers. In order to do this, we created a four-code treatment “hierarchy” in which those story treatments that required the most reporter effort were at the top of the hierarchy and those that required the least were at the bottom.
An “edited package” typically requires the most reporter effort. This treatment requires a reporter and photojournalist to spend time in the field capturing video and sound, generally produces a longer piece, and can require extensive editing time to allow for all of the relevant footage to be thoughtfully organized into a coherent story.
Next in the hierarchy came the “SOT,” short for “sound on tape.” A reporter and photojournalist spending time in the field capturing video and recording sound, usually at a single location, characterize this treatment. This type of story is typically shorter than an edited package and requires considerably less editing.
The third treatment in the hierarchy was the “VO (short for ‘voice over’) with moving images.” This treatment requires a photojournalist to capture video in the field, but does not require the capturing of sound from the field. Generally, a reporter will not accompany a photojournalist into the field for this type of story. A reporter can record sound in a studio in the newsroom to play over the video, or a desk anchor can speak while the video is playing during the newscast.
The final treatment in the hierarchy was the “reader without moving images.” This treatment requires little or no field reporting and appears during the newscast as the anchor briefly telling the viewers the story without any accompanying video.
The newscasts of BR-3 showed marked improvement in a number of important areas. The proportion of total newsroom stories with edited packages more than doubled, a dramatic increase directly affecting the quality of presentation. The proportion of BR-3’s newsroom stories that employed live shots almost tripled, another direct indication of greater effort at quality improvement. Moreover, there was a noticeable shift in BR-3’s focus of coverage, from entertaining and sensational topics to more weighty civic matters. For example, BR-3 decreased the proportion of time devoted to accidents, disasters, and sports and increased its coverage of governmental issues in first-block stories by almost 50%. In general, BR-3’s first-block story treatments reflected many of the suggestions imparted during the CCJ workshops: there was a dramatic increase in average length of first-block stories (from an average of 35.5 seconds to an average 57.25 seconds); and the proportion of first-block stories that were presented as edited packages more than doubled (from 13.3% to 32.7%). As we have noted above, both types of changes have been shown to be highly correlated with a broad range of quality indicators.
As noted above, BR-11 is a moderate sized Midwestern news station. The percentage of BR-11’s post-workshop newscast stories that were SOTs increased from 8% to 22%. The percentage of post-workshop stories that were “readers without moving images” fell from 32% to 26%. This is a positive development, as stories with more intensive treatments are generally more substantive and of higher quality.
BR-11’s post-workshop newscasts featured less time devoted to crime stories, particularly during the all-important first block of newscasts. BR-11 devoted 15% less time to first-block crime stories during its post-workshop newscasts. It devoted 6% less time to crime stories across its entire newscasts.
Despite the decreased amount of time devoted to crime stories in BR-11’s first block stories, the average length of all of its first block stories increased by nine seconds after the CCJ workshop. In We Interrupt This Newscast, Dean and colleagues found the average local television newscast story to run between 60 and 65 seconds, which makes nine seconds a comparatively sizable change.
BR-11’s post-workshop newscasts contained almost 14% fewer live shots on stories related to crime. A respondent from BR-11 noted that the BR-11 news director “put more pressure on producers to understand the stories [in their newscasts]. They [were not permitted] to just go ‘live for the sake of live’ anymore,” said this respondent.
This “live for the sake of live” comment hits on a key lesson shared during broadcast workshops. Dean and his co-trainers teach that live shots can be invaluable for offering viewers a sense of immediacy and “place.” However, they also warn that live shots can be misused to place a reporter in front of a dark building where something happened hours ago in the misguided belief that a sense of immediacy can be artificially recreated.
As noted above, BR-3 used of live shots in almost three times as many stories after its CCJ workshop as before. BR-3 posted 21% increases in its use of live shots on both government and election-related stories. While we can’t infer as to the quality of these live shots (due to the limitations of our product analysis, we were not able to see the live shots and thus make any judgment about their quality), the mere fact that they did so many more on topics that typically aren’t “live shot-friendly” offers hope that the station was using its live shots to add creativity and meaning to the telling of those stories.
While BR-3 was increasing its use of live shots all-around, BR-11 halved its use of live shots. And, as noted above, the station used live shots in 14% fewer crime stories than before the workshop. As the quote above suggests, going “live for the sake of live” may have been an issue in this newsroom before the CCJ workshop. The more sparing use of live shots, especially on crime stories – which have a greater likelihood of acquiring a “live from in front of a dark building where nothing is happening” look to them – may have been a compensatory effort to use resources more responsibly and thoughtfully after a period of misuse in service of live shots.
CCJ teaches that the live shot technique is not inherently good or bad – live shots with a thoughtful purpose and high level of execution that clearly add substantively to the telling of stories can be good regardless of story topic.
[top]
Impacts on particular elements of journalism
1. Making important stories interesting
One of the most popular modules that newspapers selected for their CCJ workshop is the engagement and proportionality module. This module proposes that a key task of journalism is to make the important interesting. Participants exposed to this module are asked to debate the supposed dichotomy between what readers want to know and what they need to know. Is there an inevitable conflict or can journalists do both? Participants are asked to review sections of their own newspaper and consider how what is written reflects the community. Who is included in their stories? Who is left out? The module offers ideas on how to make stories more relevant and pushes participants to engage in dialogue about the community they serve.
There are elements of this module in the broadcast workshops as well. Instead of reviewing sections of a newspaper, participants break into groups and must collaboratively decide how to best organize a mix of nine hypothetical stories into a newscast. These hypothetical stories are characteristic of what one might see in a typical local television newscast – an anthrax scare at the local post office, a bomb hoax at the airport, a cut in street repair funding, a live announcement by the mayor on security, a cut in education funds, a dead body found in an “exclusive” neighborhood, the threat of war in Pakistan, a medical story on coping with pain, and a runaway elephant at the zoo. The groups are tasked with “stacking” these stories in a way that reflects their significance and potential interest to viewers - this is the broadcast “stacking exercise” referred to above.
Interview respondents from newspapers, local broadcast stations, and the Internet newsroom exposed to this module consistently cited lessons from this module as key workshop takeaways.
A full 65% of the print and Internet interview respondents exposed to the engagement and proportionality module, and at least one respondent from each newsroom that requested it, said lessons from that module impacted them or their newsroom in some way. These impacts ranged from provoking staff and managers to interact with their readers more frequently; to devising systems for capturing reader comments; to writing, editing, and laying-out stories in ways that make them more relevant, creative, and interesting.
Interview respondents from case example newspapers PR-16 and PR-17 reported high degrees of impact from their exposures to this module. 88% of PR-16 respondents and 70% of PR-17 respondents said lessons from this module left lasting impressions in their newsrooms. One PR-16 interview respondent described the impact of this module in the following way: “‘Who are we impacting?’ I am now asking myself this with every story. I try to cover all the angles [and] get all the points of view.” Another PR-16 respondent said there’d been a big improvement since the CCJ workshop in the area of writing stories from a more reader-oriented and less “institutional” perspective. “[Our writing] is more engaging [and] very conversational,” this respondent said. “It makes you want to read on.” Other respondents noted that “Why should anyone care” has become a common phrase in the newsroom. One respondent said a line has actually been added to the story budget template to focus reporters’ attention on that question from the outset of their reporting.
Respondents from PR-17 suggested that some fundamental changes in the way stories were conceived and reported after the CCJ workshop grew out of the engagement and proportionality module. One PR-17 respondent described how lessons from this module changed the way stories are planned in his section. “[Now] planning involves the reporter getting an idea and working with the editor to develop a reporting plan [and] set up reporting dates and a coverage strategy. Reporters are encouraged to think about how to make ideas matter to people beyond a single story.” Other respondents described how the newsroom instituted new systems for engaging with readers, including an “email Rolodex” of community members that can be queried for story ideas and feedback and a new weekly “Community Extra” section, the purpose of which one respondent said was “to embrace the ideal of being responsive to readers [and] reflect them in the paper.” One respondent said the messages from this module caused him to “think more about getting out into the community to get the best sense I can of how people view things. I do more info gathering in the community now than I used to.”
37% of broadcast respondents cited impacts related to the parts of the broadcast curriculum dealing with engagement. Specific impacts reported include: providing “beat days” to reporters to allow them to cultivate new sources, meet with old sources, and dig up more diverse and interesting story ideas; holding “reporter meetings” for staff to share reporting techniques; and working harder to avoid cliché statements in reporting.
Station BR-3 started providing beat days to reporters and running promotions at the close of newscasts encouraging viewers to email or call the station with story ideas and feedback. “I call new people,” said one BR-3 respondent. “Now I’m consciously trying to diversify my contacts [and] get to different places.” Another respondent added the station is “attacking topics that most stations wouldn’t because they’re not very ‘sexy.’” This respondent added that in the broadcast environment, taking those kinds of risks can pay off. “If a story works out, others will follow us and run it [too].”
28% of print/Internet respondents and 29% of broadcast respondents said engagement and proportionality module lessons were the most valuable takeaway from the CCJ workshops in which they participated. One PR-16 respondent said the story brainstorming exercise and the exercise where groups dissected different sections of their paper for ways to make them more interesting were the highlights of the training for him. “[The exercises] got me so fired up to be more vigilant and less complacent.” A respondent from BR-3 added that the engagement aspects of the broadcast training taught him to not “always assume that what appears to be a big story of the day is the one that appeals to most people. How engagingly stories are written is the key to a focused newscast.”
[top]
2. Accuracy and verification
As noted above, public skepticism about journalists’ abilities and willingness to practice tools of accuracy and verification is one thing currently undermining the media’s credibility and relevance. CCJ’s accuracy and verification module introduces the idea that technology has made information easier to get and transmit, which has had implications on journalists’ traditional roles as “gatekeepers.” The module encourages journalists to examine their methods of reporting and verifying information and presses them to think about whether they are getting at “the truth about the facts,” as the Hutchins Commission put it 50 years ago. It offers concrete suggestions for enhancing credibility with readers and improving accuracy, including checklists and advice on how to handle anonymous sources.
The broadcast curriculum does not include an accuracy and verification module per se, but discussions of accuracy and sourcing are informally threaded into the engagement and news judgment lessons.
74% of respondents exposed to the accuracy and verification module, and at least one respondent in every newsroom where it was taught, cited some impact from its lessons after the workshop. These impacts frequently involved the individual or newsroom-wide use of CCJ-selected tools, including: increased transparency with staff and readers, use of accuracy checklists, use of the “skeptical editing technique,” and use of the “red pencil technique.” 25% of respondents exposed to this module said its lessons were the most memorable part of their workshop experience.
47% of respondents exposed to the accuracy and verification module reported increased efforts at transparency after the workshop. 45% reported individual or newsroom-wide use of accuracy checklists. 36% reported at least occasional use of the skeptical editing technique. And 25% reported individual or newsroom-wide use of the red pencil technique.
Being “transparent” with readers means making it clear to them how you know something – such as why an anonymous source wasn’t identified, how a source came to know some information, and what is still unknown about a story. Respondents often cited the use of editor’s columns or notes, occasional stories on how the editorial process works, and stories about how a controversial story was written as examples of transparency impacts.
“Accuracy checklists” are posted or mental checklists reporters and editors can use to help them focus on what they’ve already made sure is accurate and what may need further verifying. Some respondents said CCJ workshops inspired them to take the checklist idea further to encompass not only accuracy, but the entire reporting process.
“Skeptical editing” involves an editor going through a writer’s story line-by-line with her and playing “devil’s advocate” on any presumption or piece of information that can be questioned. A lack of time kept this from being a tool to be used on every story everyday in most newsrooms. Instead, respondents said it was especially useful on long, controversial, or complex stories.
“The red pencil technique” originated with Tom French, a Pulitzer Prize winning reporter for the St. Petersburg Times. French went through his stories line by line and with a red pencil and put a check mark on every fact and assertion. Every mark meant the fact or assertion was known or he had double-checked it. He didn’t turn his stories in until every fact and assertion had a check. Newsroom PR-23 took to this lesson wholeheartedly. During our assessment visit to this newsroom, we found signs posted in hallways and on bathroom mirrors encouraging staff to “Remember the Red Pencil Technique.”
Newsroom PR-16 modeled the internalization and implementation of accuracy and verification module tools and lessons CCJ hoped to achieve. 94% of assessment interview respondents in this newsroom said they’d changed their verification practices or noticed newsroom changes in verification practices since the CCJ workshop.
The most visible change respondents mentioned was the creation of an “accuracy and credibility group” comprised of newsroom staffers charged with analyzing and addressing corrections issues and innovating ways to keep colleagues focused on accuracy. The group developed a comprehensive staff manual that included not only accuracy checklists for each newsroom job position, but also a stylebook customized for PR-16’s specific beats and communities, a description of all newsroom policies, a “how to” guide on everything from writing obituaries to operating the newsroom computer system, and an emergency contact list and protocol.
[top]
3. Bias
The goals of the bias module are to aid participants in identifying predispositions towards certain biases and to offer strategies for identifying and avoiding inappropriate biases. Workshop participants exposed to this module are encouraged to think beyond political biases – the biases most often flagged by audiences as problematic. Workshop discussions move through political biases to general conflict of interest issues and questions of maintaining journalistic independence. The broadcast curriculum does not include a formal bias module, but ideas and lessons about identifying and dealing with biases are sprinkled throughout a typical broadcast workshop.
70% of respondents exposed to the bias module reported individual or newsroom-wide impacts from its lessons and ideas. 43% said exposure to this module was the most valuable part of their workshop experience.
Respondents at PR-16 noted that exposure to the bias module helped them increase their consciousness of subtle biases and implement checks on those biases. “[The workshop] made me really careful. Before the workshop, I didn’t consciously check myself for biases…now I do,” said one respondent, who added that probes about identifying potential biases appear in the newsroom’s new staff manual. Other PR-16 respondents mentioned that the training opened their eyes to the idea that biases can extend beyond politics.
Respondents mentioned controversial national issues such as the intelligent design debate and local issues such as a high-pitched debate over the use of new energy sources in their community as places where they needed to be more conscious of their biases. One astute PR-16 respondent added that he’d been made more conscious of what he called “logistical bias,” or the tendency to “do easier stories that require less work [and allow for the] use of sources who are easily available.”
[top]
4. Covering politics
Only a handful of newspapers have requested the covering politics module since its inception. The module discusses the purpose of political reporting, assesses participants’ own reflexes and routines, and offers a host of tools and ideas for improving political coverage. The concluding exercise invites participants to generate a new plan for covering an upcoming race. Again, there is no formal covering politics module in the broadcast curriculum, although aspects of reporting on elections are covered when relevant.
45% of interview respondents that participated in this module reported newsroom-wide impacts associated with its tools and lessons. These impacts included holding internal meetings to plan election coverage, soliciting election-related story ideas from non-newsroom staff, and trying to tell political stories in engaging ways without resorting to “canned” stories.
Newsroom PR-17 responded especially positively to its experience with the covering politics module. 26% of PR-17 interview respondents called the covering politics module’s lessons and tools the most valuable part of the workshop. “Our election coverage was more issue-focused and reader-oriented,” said one PR-17 respondent. “[The CCJ workshop] reinforced the importance of getting at what readers want to know. We surveyed readers about what was important to them.”
Another respondent added that the CCJ workshops “reinforced that we could plan to do things in a way that hadn’t been done before…be creative. We took things up a notch. Content, appearance, and delivery [of election news] was better than it’s been in the past.” In a nod to the crossover in module lessons, one respondent mentioned that the paper did a better job of “trying to maintain balance on issues and candidates,” and added that there were many fewer complaints about bias in their coverage of the 2004 election than in previous elections.
[top]
5. Watchdog journalism
Newspaper and broadcast assessment interview respondents frequently noted that one goal their managers had for improving content was to report more, and higher quality, investigative stories. The purpose of the watchdog journalism module is to get journalists thinking about subtle but important distinctions in investigative reporting, such as the differences between original investigative work and reporting on the investigations of others, or investigations into consumerism versus those of public institutions. Participants are given ideas on how to manage a major investigative project and asked to brainstorm a list of possible investigative projects. There is no formal watchdog journalism module in the broadcast curriculum; however, aspects of improving investigative journalism appear frequently in broadcast lessons and exercises.
44% of respondents exposed to the watchdog journalism module reported associated impacts. Responses featured digging deeper on stories to generate investigative angles, putting processes for generating and executing investigative story ideas into place, and other initiatives aimed at increasing the amount and quality of investigative content in their newspaper’s pages.
Newsroom PR-11 set the standard among newsrooms exposed to the watchdog journalism module in terms of the way that newsroom prepared for and incorporated module lessons. The executive editor of this newspaper told us bringing CCJ in to present the watchdog journalism module was part of a “distinct, spoken, deliberative objective to make investigative journalism a greater part of the newsroom culture.” The editor credited CCJ with “setting up the framework for the push on investigative journalism,” and remarked that setting up this framework was the most concrete impact of the training. Staff respondents clearly got the message. One remarked that the CCJ training was in line with a “greater emphasis on investigative journalism from the top-down.”
Several respondents remarked that the training reinforced that investigative elements can be incorporated into many different types of stories, and one respondent learned that investigative journalism wasn’t what he thought it was. His editor shared his story. “[One writer on our staff] saw investigative journalism only as ‘gotcha’ journalism. I didn’t know that’s how he looked at it. He learned from the CCJ workshop that that’s not how it has to be.” Importantly, the editor also noted that he learned that he shouldn’t take for granted that his staff defined certain concepts the same way he did. The editor followed up with staff that made comments during the workshop indicating that they weren’t on the same page so he could clarify things for them.
[top]
6. News judgment
Teaching workshop participants to identify the core principles embedded in the journalistic reflexes of their newsrooms and consider how they match up with each individual’s personal conscience about “doing the news” is a main goal of the broadcast curriculum workshops. Workshop participants are asked to define the “purpose” of their stations’ newscasts. Then they participate in the previously mentioned “story-stacking” exercise, where they find out how closely their reflexes match their definition of their purpose as they make decisions about how to prioritize and tell nine hypothetical stories. The idea is to refine the news judgment of each individual, and by extension each station. This module typically is only presented to broadcast audiences, and is presented at every broadcast workshop.
44% of broadcast respondents reported impacts associated with news judgment. These impacts included things such as more thoughtful story selection and placement in newscasts, more staff collaboration in developing story ideas and strategies for reporting them, and more effort to make the substance and tone of stories consistent with the station’s chosen identity.
“[The training] reinforced that telling good stories is the way to get and maintain viewers – people want substance,” said a BR-12 respondent. “[Now] we drop the stories that aren’t great. [We] learned you don’t have to do every [story]…you can just do the good stuff,” this respondent continued. This respondent added that she believed this new approach to news judgment was part of the reason for the station’s recent ratings climb.
“[The training] affected my perceptions of what is newsworthy,” said a BR-7 respondent. “You don’t have to cover every shooting. We’re rethinking viewers’ values…what viewers really want.” Another BR-7 respondent added that since the CCJ workshop there’s been “more focus on substantive stories, thinking about why we pick certain stories, [and thinking about] why they should be played where they are played [in our newscasts].”
Another BR-7 respondent credited the training with bringing about a spirit of more collaborative editorial decision-making. “[The training] made me feel like I have the ammunition to speak up about editorial decisions because [we’re doing] more collaborative decision-making. We reference the story stacking exercise in the morning meeting sometimes [and] talk about what viewers really want.”
A BR-6 respondent had a similar reaction to the training. “[The workshop] opened up opportunities to discuss story treatment decisions. [There have been] lots of ‘remember the way we discussed these things at the workshop?’ type discussions,” this respondent said. “As a team we’re more open to discussion,” said a BR-13 respondent. “We’re more able to talk about why a story is or isn’t a story. We don’t just go do things – [we] think and talk first,” this respondent continued.
One station in particular seemed to feel as though the training helped them match their news judgment to the editorial vision their recently hired news director was trying to instill in the newsroom. “[The workshop] streamlined our focus,” said one BR-11 respondent. “We think about why and how we cover what we cover [and try to] justify why we do certain stories. We look more critically at [our new judgment]. What’s the viewer value? What’s the impact? [The training] has created a momentum of thinking. There’s less following the newspaper and the wires. [We] feel better about the direction we’re taking.”
[top]
Impact on newsroom communications
Although newsroom communication was not a major focus of the CCJ training modules, it was a target of concern and a subject of many workshop conversations. It is well known in the industry that newsroom communications among and across ranks are often inadequate. Because the CCJ workshops offer participants opportunities to collaborate with colleagues on challenging tasks and to discuss jointly – often for the first time – their fundamental purposes, standards, and strategies, we expected that the experience could leave a legacy of improved communication in participating newsrooms. Moreover, since communications shortfalls in newsrooms are not only common but also highly bothersome to many journalists, we expected that participants would use the interactive workshop format as an opportunity to raise this problem and to work out better channels for communication with their colleagues. The assessments confirmed all of these expectations.
Improved communications with coworkers up and down the ranks as a consequence of the workshops was reported by 75% of all participants. This finding was virtually the same for print, broadcast, and Internet newsrooms: 72% of print, 78% of broadcast, and 74% of Internet participants reported this improvement. 55% of all participants across the three media found this to be the most valuable takeaway from their workshop experiences, overwhelming all other benefits other than learning the CCJ tools of good journalism (see section below entitled “Use of CCJ tools”).
What did participants mean by “improved communications”? Breaking down the variable into specific components, 61% of participants identified the main benefit as improved teamwork and information sharing and 24% identified the main benefit as more effectively structured lines of responsibility and, related to this, better-organized meetings. Others mentioned issues such as more frequent interactions with colleagues on off-task matters and a generally more collaborative atmosphere throughout the newsroom.
As noted above, the finding of improved communication was noteworthy because the topic of newsroom communication was not generally the main subject of the workshop modules: it was, rather, more a sub-text than a text, embedded in the way that the small-group discussions and other exercises were conducted. Informal observations by trainers confirmed the pressing need of newsrooms to pay more attention to communications gaps that undermine the working conditions of staffs. For example, in one newsroom that was visited, reporters requested that the trainers distribute nametags so that the participants could become acquainted with one another. This was a reasonable request, certainly; but it seemed somewhat surprising, because the entire newsroom consisted of less than 30 staff members, almost all of whom had been working for this same paper for well over a decade.
In another newsroom, the trainers uncovered a vast pool of discontent about the way the editor approved or disapproved investigative reporting proposals. The staff felt that the editor’s decisions were either arbitrary or based on unjustifiable favoritism. When the trainers spoke with the editor, it became clear that he had worked out an intricate, well-articulated policy on the matter. The policy reflected reasonable judgments about the reportorial experience of his staff, the particular interests of the community the newsroom served, and the newsroom’s niche within that community. The editor was convinced (wrongly) that the staff knew about this policy and understood the principles behind it. He believed that his staff knew about the policy because he had mentioned it to three or four members of his inner management team and thereby assumed that they must have spread the word around to all the others.
In the workshops’ small-group exercises, staff members who work in differing roles and levels and are forced to confront each other’s perspectives, gaining awareness of one another’s agendas and priorities. For example, just as reporters hear from the business manager about what keeps the automobile advertisers happy, the business manager hears from reporters about what keeps the credibility of the paper intact. It is not that such dialogues always lead to agreement – indeed, uniform agreement might not even be desirable – but they do lead to the kinds of mutual understandings that make for good working communication. In the case of the newsroom where the editor had failed to broadly communicate his investigative reporting policy, a follow-up visit revealed an improved sense of morale among the staff that stemmed directly from its increased trust in the fairness and legitimacy of the editor’s decisions.
Both print and broadcast newsrooms implemented improved communication systems as a consequence of the workshop training; but the broadcast communications improvements were more extensive and probably more effective than those put in place by the print newsrooms. This difference likely reflects the training’s greater impact at the whole-newsroom level in broadcast versus print newsrooms. The broadcast training format favors a whole-newsroom effect, since in broadcast stations nearly the entire newsroom eventually participates in the training: the broadcast workshop sessions are only a half-day in duration, and CCJ offers as many sessions as it takes to cycle a high proportion of newsroom staff through a training. It can be expected that this increases the likelihood of newsroom-wide changes resulting from the broadcast training. The print workshops provide intense learning experiences for the 25-40 people who participate in them; but for mid and large-sized papers, this training group may not be a large enough proportion of the newsroom to alter whole-newsroom practices in major ways.
Still, print newsrooms implemented a number of noteworthy improvements in their communication systems. These included:
1. Brown bags – A handful of newsrooms reported doing more brown bag lunches after the CCJ training. Few newsrooms began doing brown bags if they hadn’t already been doing them in the past – they just did more of them and tried to give them more focus.
2. Writer’s groups – Two newsrooms noted that they have a writer’s group. In both instances this group preexisted the CCJ training. The groups’ existence came up because both were said to have discussed relevant ideas from the CCJ training in their meetings.
3. Corrections committee – Some newsrooms started “corrections committees” consisting of rotating newsroom staff who tracked errors and met with consistent offenders to discuss ways to improve their work.
4. “Meet the copy desk” – One newsroom hosted an event so that the daytime newsroom staff could meet the nighttime copy desk staff so they could get to know each other better and develop empathy for each other’s job responsibilities.
5. Teamwork/collaboration – 57% of all print interview respondents said some aspect of teamwork, information sharing, and collaboration improved in their newsrooms after the CCJ workshop. These reports were generally anecdotal comments about the breaking down of communication “barriers” between sections of the paper, a greater appreciation for what colleagues do in their work, and in some cases a greater willingness to initiate brainstorming with colleagues about story ideas or coverage strategies.
The more extensive improved broadcast communication systems that were implemented included:
1. Reality checks – This is the “biggest” communication system change spurred by CCJ training in many broadcast newsrooms. The idea behind a reality check is that reporters and their crews in the field will call the producers of the shows they’ll be appearing in once or several times throughout the day to give updates on story progress. These progress reports ideally touch on both editorial content and logistics. Ideally, a “bridge line” is used for the calls so the assignment desk, managers, promotions staff, and anyone else can listen in and contribute, and the calls are logged in the newsroom computer system so anyone with questions about a story can simply check the log. Several stations implemented reality checks in various forms.
2. Quality control meetings – One problem many stations suffered from before their CCJ workshops was that staff would head for the door after their newscasts without discussing what went right and wrong; hence, recurring, correctable problems would not be addressed. Several stations implemented quality control meetings after newscasts – especially their main ones – where anchors, directors, producers, managers, and others could discuss the highlights and lowlights of the newscast, make plans to purge problems, and reinforce the good things.
3. Editorial meetings (“Morning meeting”) – Several station managers started approaching their morning editorial meetings differently based on suggestions from the CCJ workshop. The biggest changes were in tone, participation, and format. Managers reported trying to take a less visible role in the decision-making, instead turning that function over to assignment managers and collaborative group decisions. Some managers encouraged the participation of photojournalists, tape editors, production staff, business staff, and promotions staff to draw ideas from a more diverse audience. Some newsrooms moved away from a structure in which the manager would “go around the table” asking each participant what they had to contribute, and instead started by running through existing enterprise ideas and daily events that seemed interesting. Then they moved into whether participants had anything to add to those ideas and any new things they’d heard or been thinking about.
4. Pre and Post “Sweeps” socials – At Broadcast Curriculum Director Wally Dean’s urging to “celebrate their successes,” some stations started holding socials before and after Sweeps periods. The Pre Sweeps socials were efforts to boost morale and share a common vision for the coming months. The Post Sweeps socials were efforts to show staff their efforts were appreciated and keep the momentum going from good stories and projects that aired, even if ratings weren’t ideal. These socials also had the effect of improving newsroom collegiality.
5. “Right things done right” meetings – Two stations started doing “Right things done right” meetings at which the newsroom manager shared clips of good work from the previous days, weeks, or months and either explained what was good about them, or invited staff to share what they thought were the highlights. These meetings helped staff better understand the managers’ expectations and visions for good journalism.
6. Town hall meetings – One station started holding regular town hall meetings in all of the counties it covered to expose staff to community members’ story ideas and coverage feedback. Transcripts of these meetings were shared with all newsroom staff, and negative feedback served not only to educate staff about community views, but also to hold them accountable for their work.
7. Reporter meetings – One station’s reporters starting holding regular weekly meetings after the CCJ workshop as a way to share skills, stay on top of relevant community news, develop mentoring relationships, and bring community members into the newsroom to build new relationships and lines of communication.
8. “The easel” – Some stations started using dry erase boards or easels to log story ideas and assignments. This gave staff a simple, visible way of figuring out who would be doing what and kept good story ideas that didn’t fit into newscasts “alive” for the next day.
9. “Blue light” – One station affixed a flashing light to the assignment desk that would be flipped on during breaking news to alert staff to assemble and find out what was going on, how they could help, and how their previous assignments might change.
Both the print and broadcast improvements were promising and, certainly, constructive; but, as noted, the broadcast changes were more sweeping, innovative, and transformative than those that we observed in the print newsrooms.
[top]
Newsroom innovations
Traveling Curriculum workshops offer journalistic tools and the inspiration to think critically and communicate more openly with colleagues. Another encouraging impact discovered during assessment interviews is that workshop participants are using CCJ tools, lessons, and ideas as the foundation for their own initiatives and innovations. Individuals in nearly 80% of all print, broadcast, and Internet newsrooms assessed described innovations in their newsrooms they attributed directly or indirectly to their CCJ training experiences.
At newspapers, the innovations ranged from forming “accuracy and credibility” staff committees, to creating “reader panels” to critique previous work and offer story ideas, to sending letters to sources to ask if they felt like they were quoted accurately and fairly, to initiating newsroom awards programs, to “how I did the story” brown-bag lunches to share reporting techniques.
At local television stations, innovations ranged from inviting non-newsroom staff to editorial meetings to offer story ideas and feedback, to inviting viewers to share digital photos to the stations’ websites and newscasts to make them feel more a part of stories, to structured story filing protocols aimed at increasing accuracy and creating accountability.
Non-medium specific innovations included: reader/viewer databases that can be queried for story ideas, leads, and feedback; community meetings to discuss controversial editorial decisions and make connections between staff and the community; and holding “diversity workshops” where groups of people representing various demographic groups in a community are brought into the newsroom to meet staff and discuss their ideas and perspectives.
Interestingly, the assessment visits spurred ideas for innovations as well. One newspaper started cycling all editorial staff through small-group meetings with the executive editor to give them opportunities to discuss editorial decisions, ethics, and the editor’s visions for the future after the assessment visit. The report sent to the newsroom after the assessment interviews recommended creating more opportunities for staff to interact with upper management and each other. One broadcast station started discussing how they might conduct “pre-show” meetings before newscasts to give directors and producers a chance to go over the details of the newscast and thus avoid preventable errors. This discussion was spurred by the assessment team observing through interviews that preventable errors were a problem and recommending that staff formalize ways to deal with them.
[top]
Training, Strategic Planning, Critical Thinking
Powered by: Phase2 Technology