Does structured journalism work? Evaluating the feasibility of structure for consumers and reporters

Does structured journalism work? Evaluating the feasibility of structure for consumers and reporters

Does the traditional news article still make sense as the primary unit of news in the age of the Internet and smartphone? That was the question I asked in my first blog post as an RJI Fellow in July 2015. My fellowship has focused on evaluating a proposed alternative unit of news: the structured story, a knowledge representation mechanism that enables the encoding of general news as structured events organized into narratives.

Structured stories are an extreme example of structured journalism, a broader movement that seeks to change the basis of journalism from unstructured text articles into a data-centric, networked vision of news. Such a vision might offer a sustainable path forward for newsrooms, news consumers, journalists and society in the challenging post-print media environment.

Over the past year I have worked with small teams at the University of Missouri School of Journalism and at Duke University to answer two key questions about structured stories:

  1. Is it feasible for real journalists to report real news as structured narratives?
  2. Is there real value for news consumers from news conveyed as structured narratives?

A year ago these questions had only speculative answers supported by anecdotes. My goal during my fellowship was to systematically investigate these topics, guided by professional journalism researchers. I wanted to develop an academically credible understanding of the feasibility of the approach — grounded in data — and to communicate that understanding within the computational journalism community and beyond.

Exploring the feasibility of reporting into structure began early in the fellowship with a major reporting project in New York City funded by the ONA Challenge Fund and led by Bill Adair of the Reporters’ Lab at Duke University.

Two other reporting projects were conducted in the fall of 2015: one by a team of undergraduate and graduate students at the University of Missouri, led by Ph.D. candidate Frank Russell and funded by RJI, and a smaller one by several reporters at a major TV newsroom in Southern California. These projects, in total, involved about 10 reporters. Although a few struggled with the abstraction necessary for structured reporting, others quickly adopted the approach and used it productively. These projects generated more than 100 separate structured stories, and were thoroughly documented through direct observation and using a series of interviews and questionnaires. While the projects revealed serious challenges and significant flaws in the platform and in workflows, they also clearly demonstrated that reporters can, in fact, successfully individuate real-world news events and report them directly into fully structured narratives.

Structured stories are an extreme example of structured journalism, a broader movement that seeks to change the basis of journalism from unstructured text articles into a data-centric, networked vision of news.

Exploring the value of structured stories to news consumers started later in the fellowship, and required the development of a new much-simplified user interface to use as stimuli for formal user surveys. These surveys directly compared understanding and trust available to consumers from a series of structured stories and information-equivalent text articles. The surveys were first applied to students at the University of Missouri, then to a much broader sample of news consumers nationwide using Amazon’s Mechanical Turk labor marketplace. The results of these surveys have yet to be processed; however, preliminary indications are promising. Comments solicited from survey respondents appear to indicate that structured narratives are judged to be easier to understand, although whether that is in fact the case remains to be seen.

These are illuminating and promising conclusions, although still preliminary and still not fully understood. My research partners and I have already published some of our detailed results in several peer-reviewed academic papers and will publish more in an upcoming paper to be delivered at the AEJMC CTEC conference in August in Minneapolis. We also have three academic papers planned for submittal over the next six months, including those detailing the consumption study and its results.

Other academics are also publishing papers analyzing the structured stories project, including C.W. Anderson, who is writing a pair of papers on the ethnography of the New York City reporting, which will be published next month at the ICA conference in Japan. In addition to academic publication, I’ve also used my fellowship to publish (right here on the RJI blog) a four-part series that systematically reviews the potential of structured journalism to address systemic problems facing newsrooms, news consumers, journalists and society. The Structured Stories project is, foremost, a proposal of an alternative paradigm for journalism, and proposals must first be communicated in order to be considered.

A year spent evaluating Structured Stories leads me to conclude that the approach is indeed feasible; however, a feasible approach is not necessarily a practical approach. The next phase of the project will focus on practicality, including:

  • Building new mechanisms for easier reporting and editing.
  • Building more sophisticated consumption products.
  • Implementing workflows better suited to day-to-day use.

Niches in which the technique can most likely provide early value include business-to-business publishing and local and state government coverage — both with tangible demand for efficiency of reporting and for innovative ways to manage and communicate complex narrative information.

Structured Stories works, albeit in a simplistic way, and is therefore poised to transition from an experiment into a product. Making the platform practical enough for regular use could unlock continual user-based feedback, leading to further user-directed improvement. That won’t be easy, but it will be fun. If you are interested in learning more, or would like to be involved, please get in touch with me.

Comments

Comments are closed.