Machine Learning & Essential, Actionable Insights for the Publishing Industry

Machine Learning & Essential, Actionable Insights for the Publishing Industry
What we can learn from advanced algorithms and what they hold for the future
BY MONICA LANDERS, CEO OF AUTHORS.ME

We began this company as a standardized solution to the laborious and inefficient methods of the traditional query process which is often painful for individual authors as well as publishers and studios. We’ve evolved this platform into a breeding ground for dynamic, transformational publishing technology that benefits every part of the industry. More than two years out, we have developed exciting technology that forecasts successful projects. After many conversations with industry professionals, we are more confident than ever that it can be changed for the better through technology and the essential insights it stands to receive.

Data-Driven Operations and Collection
Our platform is not only robust, it’s extremely effective.

Without getting into how the platform works (you can learn that here, here, and here), let’s look at the state of the industry’s submissions and publication statistics. According to writer Joseph Epstein, at any given moment 200 million Americans have a book they want to publish.  Digital Book World  surveyed writers and discovered that more than 60% submitted their work to a publisher or agent the previous year.

From that, we can estimate that anywhere between 125 million writers submit manuscripts to publishers and agents in the US every year (though anecdotal evidence from agents may suggest more like 20 million submissions every year3). This means each publisher/agent is receiving  between 3,000 and 20,000 submissions a year1. So, the likelihood that a given submission will be published is between just .25%2 at the conservative end and 15% at the optimistic end and, more importantly, the likelihood a manuscript will be rejected or ignored is up to 99.76%  

It’s a wonder anyone tries at all. But, the fact that they do means that there is real worth in trying to make the system work better.

With the AUTHORS.me platform, writers are 7 times more likely to be accepted and 13 times more likely to get positive, forward movement for their manuscript.  

For many writers the most frustrating part about the submissions process isn’t being declined.It’s not knowing where you stand or what is going on. While it is an ongoing process to get publishers and agents to use our workflow statuses to accurately represent when they review work, 39% of writers who submit through us know concretely that their work has been declined and just 56% in the lifetime of the platform are awaiting review. Considering that lifetime submissions on the platform nearly doubled in the past three months, that is a true improvement to the norm.

Transformational, Dynamic Development
Algorithm Progress & Essential Insights

For the past year and a half, our developers have been honing and deepening a patent-pending algorithm that delivers the probability an individual manuscript could be a bestseller. The underlying goal is to offer a product that helps publishers, agents, and production companies identify and act on lucrative properties more quickly and with increased acuity. Our belief is that this technology offers the industry transformational power driven by actionable insights.

As the work went on, we discovered that beyond just that singular determination, the program was able to identify strengths and weaknesses within a given piece of writing that could, by an editor or a writer, be turned into essential, actionable insights that both expedite and strengthen the editing process and go-to-market plan.

For example, a report may detect room for improvement in areas such as redundant phrasing, incomparable constructions, or explicit language use. Editing with specific actions or recommendations is far easier and less overwhelming. A manuscript that seemed like it just wasn’t working now has a dynamic road map for revision.

Likewise, the sentiment analysis and comparable literary archetype can, to an industry professional, become a keen market insight that allows for a faster, more objective method of finding comparable titles and, informed with a title’s less obvious but no less essential common characteristics, possibly expand the target audience. In an industry with a reputation for homogeneity both in representation and delivery, these kinds of tools bolster objectivity and in turn create a more diverse landscape.

With these insights in mind, we launched the first iteration of our Intelligent Editorial Analysis Reports in partnership with BookLife, a Publisher’s Weekly website that seeks to provide self-published authors with resources, community, and platform elevation.

The report uses the technology we have been developing for publishers and enterprise entertainment companies and delivers digestible, actionable insights on an individual manuscript. Anyone can upload a manuscript and receive feedback on elements of their writing from style and grammar to syntax and literary device implementation. It points out areas for potential revision as well as commendations for markers of “good” writing. It shows the writer the manuscript’s literary archetype based on sentiment analysis, and it also delivers a numerical evaluation of their manuscript in comparison to best sellers.

The road to get here was full of curious, fascinating experiments and realizations. Let’s look at a few of them.

 Archetypes

We’ve analyzed thousands of books—bestsellers, mid-list titles, backlist classics, self-published books, and unpublished manuscripts—to develop and improve the algorithm. One of the measurement points is a sentiment analysis, which when translated into a plotted arc resembles the narrative arc of the story in question. In performing these thousands of calculations, we discovered that there are measurable differences in manuscripts down to the tenth decimal point.

That is how unique each piece of text is, and how quickly and easily a computer can prove it. Within those fractions of variation exist essential insights into tone, character, style — the possibilities for measurement and action are boundless and thrilling to data scientists and forward-thinking literary analysts alike.

Sentiment Analysis — Part of a Whole

Many teams are working on understanding NLP better and we’ve been able to incorporate into our systems training some of the smartest APIs available., such as Microsoft’s Watson program. In one of these tests, we asked Watson to analyze the sentiment arc for particular  parts of a book; specifically different characters and settings (it’s fascinating but not altogether surprising that an element can have a narrative arc, but that’s a story for another day.)

When we ran these results back through our own program and analyzing bestsellers, we found that the overall sentiment arc plays a much larger role in determining a title’s comparability to the standard bestseller profile than that of any one part of the book. Or, in simpler terms: a book is more than the sum of its parts. The romantic part of me somehow thinks we all knew this already, but this is documented, objective proof to that effect.

Data & Publishing
Finding Meaning in the Numbers

The larger point is that in all of this work, we are discovering objective key performance indicators of raw text that transform previously elusive, ephemeral qualities of writing into quantifiable, measurable, and meaningful data points.

With this information, editors and writers alike can optimize their own individual approach to their work. The industry and community at large can harness the raw power of Big Data to stake claim to their creativity and carve out pieces of the market that fit best, not just fit now.

We’re seeing that the increased prevalence and use of data in publishing doesn’t have to mean a withering competitive landscape, but instead a richer, more vibrant one where the bar is continuously raised, met, and transformed altogether.


1 There are roughly  6,080 traditional publishers and agents in the US.  [2014 SUSB Annual Data Tables by Establishment Industry]

2 In 2016, 311,723  books were traditionally published in the US. [International Publishers Association]

3 In our initial R&D, polled agents and editors who accepted unsolicited work reported an average of 100 submissions/week. 

Monica Landers

Monica is the CEO and co-founder of AUTHORS.me.

Leave a Reply

Your email address will not be published. Required fields are marked *