testing creative: how it sharpens and how it dulls the idea

—————

“It is easy to dilute a good creative idea. It needs someone who understands the idea and is a guardian of it checking back that we are delivering on it,” she says.

There is also evidence to suggest too much testing can be a bad thing. Further research by Field found a negative correlation between the use of quant in pre-testing and the success of IPA award entries.”

Kerry Chilvers

(Marketing Week: Measuring the magic: Why brands need to refocus on the effectiveness of creativity)

———————–

 

Ok. In January 2011 I wrote this piece but the following quote, and article, in Marketing Week made me pull it back up. Part of being in an advertising agency is the infamous discussion about testing creative before it is produced.

Testing the actual creative concepts and ideas in other words. Let me begin with two thoughts:

  • “The way in which advertising influences customer choice is not a tidy, mechanistic process.  It is an extremely untidy, often irrational, human process.” (Jeremy Elliott, JWT)

  • Ongoing creative testing has a habit of dulling sharp ideas which can be developed in concise upfront testing

Look. Everyone in business is in the business to create great work. I say this regardless of whether you work in any creative communications company or not.

We all want to create great things.

 

But.

 

We don’t want to be known as the people who create brilliantly crafted failures.”

Great work builds client’s brands and sells client’s stuff.

Great work makes consumers sit up and pay attention enough to, well, pay attention & do something.

Great work makes clients money through greater ‘full revenue’ volume <that’s “selling at the highest value”> and it makes us proud to say, “Yeah, we did that” (both at the same time).

Yeah. Great work is all that. So when someone offers some simplistic fortune cookie wisdom about “what great creative work is” you may want suggest it is a little more complex than what they are saying.

With all that said, talking about testing creative ideas tends to drive me crazy.

No. The consumer should never be neglected.

But. Nor should they be empowered to tell us how to say, or do, something. I believe in using research to inform decisions, not to make them.

We should use consumer research at the beginning of the whole advertising process to find out what to say, not how to say it.

Period. Stop.

Is that extreme? Yup.

Does that mean I don’t believe in quantitative research on creative? Nope.

I just tend to believe valuable quantitative creative research is the exception. The majority of time creative testing dulls great sharp creative ideas. It smooths them out to be less extreme.   And in today’s world being gray may make you feel better, but it won’t generate the interest and results needed to break through a challenging fragmented world.

Now. I don’t want to confuse creative testing with positioning research. Depending on the scope of the project strategic development research or positioning research done in innovative sometimes non-traditional thinking ways can uncover the best way to truly find out what the potential customer thinks.

A research plan of action should do something very simple very well. Talk to the inner brand and customers and consumers.

The information received at this stage forms the objectives of the work, the key idea to be communicated, sometimes an insight <either directly given or indirectly inferred> and to maybe set some guidelines on the right brand personality.

But you also have to mix what you learn from these conversations with what we know about the client’s marketing problems, and the dynamics of their business (that is a subjective component).

Using all this observation, all this hard and soft research, as well as our own personal professional experiences <insights gained elsewhere>, we then should turn it into creative insightful thinking.

We should be synthesizing some core truths about the brand into relevant consumer insights. These research-guided truths become the foundation for many creative solutions.

Oh. We can also use research to find out if we are saying what we intended to say.

You can call this “creative development research.”

Yes, this is evaluative research, but the purpose here is only to develop and nurture work, not to kill it. Not to “ask permission” of customers or clients to go with the work.

Although I do not, in principle, object to using research to inform creative decisions … in the end I struggle with some research because I do not believe in using research alone to judge the validity or effectiveness of the work.

Why?

Because we, not just me, know that the critical deconstruction that takes place within research just doesn’t happen with exposure in real life.

I guess I get frustrated sometimes because if we’re not careful the research will test the quality of stimulus rather than the quality of the ideas. It will test the quality of the presenter, or the quality of the drawings on the storyboards, or any number of irrelevant details. It will test details more often than definitions.

In other words … just relentlessly seek to allow the ideas to keep the ‘sharp edges’ that make them interesting and stand out. We should always seek to protect the ideas (big, medium or small) and give them the best chance of survival, to keep the work from being” pecked to death by ducks.”

 

The simple fact is that bad research kills good ideas.

The more complex fact is that too much good research can also kill good ideas.

Ok. Here are some things to remember:

 

+   Test materials don’t have the magic of finished commercials.

+   Commercials that have a familiar feel often “score” better than commercials that are unique, strange, odd or new.

+   Disagreement in people’s input can be a good thing, because great ideas are often polarizing.

+   We should not take what consumers say literally.

+   Remember, we use research to inform our decisions, not to make them.

+   Do not let groups of people in qualitative research settings become Copywriters and Art Directors.

+   Clinical research settings often produce different responses than cultural/anthropological research <bars, malls, street, homes, cars, etc>.

 

What would I do if pushed into a corner and said research had to happen with creative <not positioning>?

  1. Use qualitative (I do lean toward online focus groups these days) to inform on wording and ideas and claims and stuff like that.  No storyboards or creative concepts involved. Use this to gain nuggets of knowledge with which to use as the creative ideas are developed.
  2. Use quantitative (if necessary at all) to isolate which executions are most effective in communicating awareness/likeability/intent to act/specific understanding (of some specific element).  I lean toward some interest scan or MillwardBrown Link testing simply because it can inform you of some possible specific communication obstacles’ within an execution which if you are open to ‘fixing’ can improve an overall score. The most expensive way to do this is with finished executions (which some clients are open to) and the least expensive way is story board format (rips in the middle).

There you go. A lot of people will disagree with this.

But communication agencies … heck … all businesses … get judged by the quality of their effective work.

Yes. Quality of work and effectiveness.  Combined.

Research tends to make the creative more mediocre. That is bad.

Which then tends to soften possible sales/purchase spikes (you miss out on the higher highs).  That is bad.

 

Let me conclude by saying this is actually about physics (in a way).

The sharp knife exerts a large pressure on the surface due to the small area of contact. In other words. The sharper the idea the easier it cuts through clutter and the less money it takes to make the idea noticed.

The duller the idea the more effort it takes to cut through the rest of the stuff out there (and into a consumer’s mind).

I don’t know if that is a postulate, but I do know I can prove this in my own kitchen with a dull knife. Its just physics, stupid.

Bottom line? In today’s business environment it seem like we should be seeking less opportunities to play it safe and more opportunities to smartly stand out. Let me say this another way. We need more smart sharp ideas and work.

 

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Written by Bruce