The price of conviction
Why a seminar costing several thousand euros, a hall in a trance and overwhelming testimonials prove nothing about a method's effectiveness, and what would prove it
0. Introduction: convincing is not working
A vast hall, music that makes the floor shake, thousands of people standing and chanting a word, and on stage a radiant figure who promises to change your life in three days. You come out transformed, certain you have lived something rare, and the price paid, sometimes several thousand euros, suddenly seems trifling next to what you have just felt. It is precisely this feeling that must be examined, not to scorn it, but because it serves as an argument: if it was so expensive, if everyone believes in it, if so many lives have changed, then the method must work.
This chain of reasoning is natural, and it confuses two things that must be kept apart. A high price and an intense fervour prove neither the value nor the effectiveness of a method, because a convincing experience is not the same thing as an effective method. To find one’s bearings, one sorts what is claimed into two families: the feeling of transformation, assessed by asking whether it would be there anyway, and the factual claims about value, of the kind price equals value or success equals effectiveness, which are checked head-on. The distinction looks abstract; it is decisive in practice. For the feeling, the right question bears less on its sincerity, almost always genuine, than on what triggers it: would it be there anyway, by the sole effect of the staging, the price and the expectation? For the factual claim, on the contrary, no meta-test holds: if one asserts that a method works better than another, that is measured, and only a comparison can settle it. To confuse the two families is to treat an emotion as a piece of data, and that is the error the whole machinery of the seminar is built to encourage.
Before criticising anything, one must take the offer in its strongest form. These premium programmes are sometimes intense, mobilising experiences, genuinely memorable for those who live them, and it is that version, not its caricature, that must be examined. A simple tool helps to do so: the same arguments, the price, the crowd, the testimonials, the success, would establish just as well the value of any rival guru or competing programme, so that they establish that of none in particular.
One must also clear up a misunderstanding straight away, out of honesty. Explaining the success of a figure like Tony Robbins by the psychology of persuasion does not establish that his method is without effect, for some clients genuinely progress; this text does not conclude that it is a scam. The point is more precise, and the scope must be too: we clinically evaluate no programme, we measure the proper effectiveness of no method and we do not settle which kind of support works, we examine the signals that are taken for proofs of value. The aim is not to expose a fraud; it is to give each thing its due weight, by separating the force of the impression from the solidity of the proof, so as to decide in full awareness rather than in a state of excitement.
1. The price would prove the value
Let us begin with the price, since it is what opens the door. The idea that paying a lot guarantees quality is deeply rooted, and several mechanisms explain it without any of them proving the effectiveness of the content. The first is the sunk cost, coupled with effort justification: once a large sum is committed, one clings to what one has paid, and the more the entry cost, the more one values what it gives access to, at nonetheless constant content, like the person who, having paid a very dear seat for a disappointing concert, convinces himself he had a good evening so as not to admit a wasted expense (Arkes and Blumer 1985; Aronson and Mills 1959). The seminar costing several thousand euros triggers the same reflex, more powerfully: the size of the sum makes admitting disappointment too costly, and the mind prefers to revise upward the value of what it received. To this is added signalling theory: a high price signals quality without guaranteeing it, which is enough to make it attractive without establishing any link with real effectiveness (Spence 1973). The price does not merely attract, then, it buys in advance part of the satisfaction one will draw from it.
The most striking point is doubtless the placebo effect tied to price. A placebo presented as expensive relieves more than a cheap placebo, an effect found in fields as distant as pain and Parkinson’s disease, where an inert treatment said to be costly improves motor function more than an inert treatment said to be cheap; the felt improvements are real, sometimes measurable right down into the body, but they stem from the expectation triggered by the price, not from a specific content (Waber et al. 2008; Espay et al. 2015). On pain, an experiment that has become classic shows it well: a tablet with no active ingredient, presented as a costly painkiller, relieves a clear majority of the volunteers, whereas the same tablet announced as discounted relieves only a fraction, though nothing distinguishes them (Waber et al. 2008; Espay et al. 2015). The price does not bias only the judgment, it modifies the experience itself: the same wine tasted as expensive is judged better and activates the brain’s pleasure regions more than the same wine tasted as cheap, a sign that the price enters into perception, and not only into the judgment made afterward (Plassmann et al. 2008). These results converge from one study to the next, even if they sometimes rest on small samples, which invites handling them with the same caution as the rest. Transposed to the seminar, this means that part of the lived effect is produced by the price, and would recur with identical content sold for less.
Other springs add up as the sum grows. The effort and money invested increase attachment to the outcome, a phenomenon called the IKEA effect: one loves more what one has helped to produce, independently of its objective quality, and a programme demanding in time, money and effort will be valued for what it cost as much as for what it brought (Norton et al. 2011). Premium schemes have understood this well and load the journey with efforts to furnish, exercises before dawn, notebooks to fill each evening, gruelling role-plays in front of the others, each pain consented adding to the attachment a share the content alone would not have produced (Norton et al. 2011). The first figure announced, often very high, serves as an anchor: presented after a tariff of several tens of thousands of euros, a seminar at a few thousand seems reasonable, by an anchoring bias that operates even with arbitrary numbers, a robust result although sometimes drawn from small samples (Tversky and Kahneman 1982). The bias is so crude that a number bearing no relation to the question asked is enough to pull the estimates that follow, as in the experiment where a figure drawn at random in front of the participants then steers their assessment of a magnitude that has nothing to do with it (Tversky and Kahneman 1982). Lacking any other benchmark, buyers finally infer quality from the price, so that an expensive product is spontaneously perceived as better, without the slightest information on its effectiveness (Rao and Monroe 1989). Placed before two offers about which he knows nothing, the buyer almost always leans toward the more expensive, supposing it superior, and the inference holds even when the two are in reality identical (Rao and Monroe 1989). None of these mechanisms says anything about what the method really produces; all steer the evaluation before the content has even been tested.
The sale itself is organised to push toward the expensive offer. One presents first a very high tariff, then an unattractive intermediate option, then the offer one wants to sell, and this simple arrangement shifts the choice toward the latter by a decoy effect, unrelated to its real value (Huber et al. 1982). Full access displayed at fifteen thousand euros, an intermediate formula at nine thousand that offers almost nothing more than the entry level, then the targeted offer at ten thousand that suddenly seems the only reasonable one: the middle option serves only to make the target stand out, never to be chosen (Huber et al. 1982). Once the sum is paid, escalation of commitment pushes one to defend the expense rather than question it, so that the one who paid the most is also, often, the least inclined to consider the possibility that it changed nothing (Arkes and Blumer 1985). The most committed often becomes the best advocate of the offer, quick to recommend it around him, for conceding the uselessness of the sum paid would amount to admitting a mistake all the more galling for having cost dear (Arkes and Blumer 1985). One must concede that part of this value is real: for some buyers, the high price is itself the appeal, it sells a belonging and a distinction that genuinely matter to them, while remaining distinct from the effectiveness of the content. To wear the badge of the most exclusive circle, to rub shoulders with people who have paid the same price, to tell oneself a member of an elite that has treated itself to the best: this appeal is authentic and is bought knowingly, owing nothing to what the method then produces. One can go further in the concession: for some, this high price is precisely what obliges them to invest themselves and do the work, and the gain that results can be quite real, without being proper to the content sold for all that, since a lesser commitment would have mobilised less. At this stage, the price no longer only selects the buyer, it shapes the judgment he will then pass on what he has acquired.
2. The fervour of the hall
Next comes the hall, and its collective energy. What one takes for the proof that something true is happening is largely the product of a well-documented engineering of the crowd. Emotional contagion propagates affective states from one neighbour to the next: enthusiasm spreads through a hall like a wave, everyone vibrates because the hall vibrates, and reads in it the proof that something extraordinary is happening, without reasoned conviction (Hatfield et al. 1993; Barsade 2002). It is enough for a neighbour to stand up in tears or raise his arms for the surge to sweep the row, then the block, then the whole hall, everyone tuning to the emotion of the others without even noticing (Hatfield et al. 1993; Barsade 2002). To this is added exhaustion: long days, little sleep, skipped meals, dancing and shouting sustain a state of arousal and fatigue in which critical thinking lowers its guard, in which the brain processes information by its most economical route, that of surface signals and shortcuts, rather than by the calm examination of arguments (Killgore 2010; Petty and Cacioppo 1986). A typical day stretches over twelve hours or more, standing, jumping with arms raised to the beat of saturated music, the lunch break cut short or removed, sometimes a walk over hot coals meant to engrave in the body the idea that one has just tamed a fear (Killgore 2010; Petty and Cacioppo 1986). In this disposition, an assertion launched with aplomb and taken up by thousands of voices seems obvious without having been weighed for a second; what carries conviction owes less to the quality of the point than to the state of the one who listens to it. The seminar is in no way dishonest in this, it simply organises the conditions in which one believes most easily, and that deserves to be kept in mind so as to tell apart, the next day, what one understood from what one merely felt.
Group pressure also acts on the judgment itself. When thousands of people rise, applaud and repeat a formula, the individual who would doubt finds himself facing a unanimous majority, and this conformity can bend his judgment, sometimes against what his own eyes tell him, an effect since confirmed and qualified by an international meta-analysis covering dozens of studies and some twenty countries (Asch 1956; Bond and Smith 1996). In the original protocol, participants ended up designating as equal two lines of visibly different lengths, for the sole reason that the whole group, in on the trick, had asserted it before them (Asch 1956; Bond and Smith 1996). That everyone seems to adhere then works as a proof in itself, a social proof that dispenses with examining the substance, among the best-established levers of influence (Cialdini and Goldstein 2004). Seeing thousands of people already standing and convinced works as a shortcut: since so many people adhere, one tells oneself the demonstration must have been made somewhere, and one spares oneself from making it again (Cialdini and Goldstein 2004). The authority and prestige of the figure on stage further increase adherence to his instructions, as the social psychology of obedience showed long ago (Milgram 1963). Famous experiments established that the mere presence of a figure perceived as legitimate pushes ordinary people to carry out instructions they would never have followed on their own, so much does the costume of authority suffice to win assent (Milgram 1963). And this figure, haloed by his displayed success, enjoys a credit that nothing in his words necessarily justifies, by a halo effect that passes charisma off as competence (Nisbett and Wilson 1977). An assured bearing, a voice that carries, a displayed success, and the audience grants his advice from the outset a soundness it has not verified, as one supposes someone competent for the sole reason that he seems sure of himself (Nisbett and Wilson 1977).
All this composes a device, and a device has a function. The staging, music, light, collective movement, charisma, acts on the state of the audience, not on the solidity of the content, and one would be wrong to take the intensity of the experience for the quality of what is transmitted. The prolonged immersion, several days, long hours, a break from daily life, further intensifies adherence by acting on the participant himself, independently of what he is taught. Three days cut off from work, family and telephone, plunged from morning to night into the same narrative and the same rituals, inevitably leave a mark, whether the point made is sound or hollow. The sale, moreover, does not wait for the end of the seminar, it unfolds during it. One announces limited places for the higher tier, a discount that expires that very evening, an opportunity that will not come again, and this manufactured scarcity increases the appeal and triggers the purchase independently of the value of the product (Cialdini and Goldstein 2004). A countdown appears on the screen, a voice repeats that “only a few places remain” and that “the price goes back up at midnight”, and the urgency pushes one to sign before having compared anything at all (Cialdini and Goldstein 2004). One also brings each person up onto the stage of public commitment, has him declare before the hall what he is going to accomplish, and this word given before witnesses binds him to his decision, by a need for consistency the device knowingly exploits (Cialdini and Goldstein 2004). One is asked to stand, to turn to one’s neighbour, to say aloud the goal one commits to reaching, sometimes to write it and sign it; having declared oneself in this way before the hall makes turning back more costly than carrying on (Cialdini and Goldstein 2004). The sequence is well-honed: bring the emotion to its peak, then propose, in that precise state, to commit further.
3. The transformed lives
There remain the testimonials, which seem the strongest argument because they are embodied. They deserve to be taken seriously, and to have what produces them examined. One rarely signs up on an ordinary day, rather in the trough of a wave, after a breakup, a failure, an exhaustion, that is, at the moment when, statistically, things can hardly do anything but get better afterward, seminar or not; the improvement that follows, attributed to the method, would for a part have come anyway, by simple regression to the mean, while one does not see those who relapse or drop out (Barnett et al. 2004). Someone who books a three-thousand-euro seat the day after a divorce or a dismissal starts from a rare low point, and his simple return to the usual level, a few weeks later, will take on the look of a spectacular recovery that he will credit to the seminar (Barnett et al. 2004). Expectation and the placebo effect moreover produce a felt improvement independent of the content taught: believing that it will work takes part in its working (Greenberg et al. 2006). To approach the following weeks with the conviction of being henceforth equipped changes the way one acts, dares, perseveres, and this share of the improvement flows from the expectation itself, before any particular teaching (Greenberg et al. 2006). To this is added the asymmetry of what one sees: one hears the dazzling successes, brought on stage and shared, and one does not hear those who dropped out, relapsed or changed nothing, because they do not testify, so that the picture is a sorted sample in which only the successes speak. Out of a thousand enrolled, one hears the ten who go up on stage recount their metamorphosis, never the hundreds who have resumed their former life, for lack of a microphone and for lack of any wish to testify to a failure. Once the decision is taken, the feeling confirms itself on its own: having paid and chosen to believe, one notices what validates that choice and lets slip what contradicts it, so that the impression of effectiveness swells afterward without any new fact requiring it (Nickerson 1998). The slightest good week becomes the proof that the method works, a bad month is filed under circumstances, and the assessment one draws up leans toward the side one had already chosen to believe (Nickerson 1998). The mere fact of knowing oneself committed, observed, held to account or bound to note one’s progress is enough, finally, to raise the effort for a while, what is called the Hawthorne effect, and the short-lived improvement that results is readily credited to the method rather than to the attention one is the object of (McCambridge et al. 2014). The person who notes each morning his mood and his three priorities, knowing that a mentor will reread them, gives more of himself for a few weeks; it is the attention received that mobilises him, more than the grid itself, and the improvement observed ends up attributed to the programme (McCambridge et al. 2014).
Research on support sheds light on where the real effect comes from. When one compares very different approaches, one finds a largely shared effect, which owes less to the proper technique of each than to common factors, a relationship of trust, a positive expectation, a mobilising frame, the feeling of being supported; these factors explain a large part of the results, so that a simpler and far cheaper device, provided it gathers the same ingredients, can produce comparable effects (Wampold 2015). A weekly discussion group, a follow-up notebook, a close friend to answer to and a clear deadline already gather the essentials of these ingredients for a trifling sum, which places the surplus paid on the side of the experience more than of the result (Wampold 2015). The mechanisms of the placebo, expectation and conditioning, even produce measurable physiological effects, proof that a real improvement can occur without a specific active ingredient (Benedetti 2008). That the improvement is real is therefore not in question; what is, is the idea that it would come from a secret knowledge reserved for those who pay the high price, when the levers that produce it are known, public and cheap.
Two confusions remain to be cleared. The great thrill of the seminar, the breakthrough one recounts afterward, is an intense emotional state, which does not amount to a durable change of skill or behaviour, and to take one for the other is to overestimate what will remain. To weep, to shout, to collapse then rise transformed makes a striking memory, which says nothing about the habits actually kept once back home, alone, without the hall or the music. Lacking follow-up at a distance, the momentum felt on Sunday evening says nothing of how it will be three months later, when it will have dissipated, and this decline appears in no testimonial, since one testifies while still warm. In any case, transformation testimonials are produced by every method, gurus, rival programmes, therapies, traditions, so that they do not sort out their respective value. The same accounts of turned-around lives are gathered at the exit of a silent retreat, a gruelling sports camp or a religious conversion, which makes them powerless to designate which of these paths would carry an effect proper to it.
4. The investment and the network
A more subtle argument deserves a separate examination: the correlation between having followed these programmes and later success. One readily presents the list of participants become prosperous as the proof that the method leads to success, whereas paying several thousand euros and blocking a week of one’s schedule already supposes uncommon means and determination: the price operates a socio-economic filter, and the correlation reflects a selection before any causal effect of the method. Freeing five full days in the middle of the week and pulling several thousand euros from one’s pocket rules out from the start the greater part of the population: the hall is nothing like a representative sample, it is sorted by the wallet before the first word spoken on stage. The hall is therefore, from the outset, filled with people more affluent and more willing than average, whose later success may flow from these prior qualities, the very ones that led them to enrol, rather than from what they heard on stage. To untangle the two, one would have to compare these people with others, equally determined, who would not have followed the programme, and it is this comparison that is lacking.
The content, moreover, is not always the real product. Often it is the network, the famous mastermind, that creates the value: what the buyer obtains, and which has an authentic price, is access to a circle of ambitious peers, contacts, opportunities, a feeling of belonging, a connection value distinct from that of the content taught. It happens that the true value of a year in the circle is decided in a corridor, an introduction made between two workshops, a partner or a first client met there and nowhere else; this value is quite real, it belongs to the address book gathered more than to the point made on stage. It can justify the price on its own for some, provided one sees clearly that one has paid to enter a club, which can be a good calculation, and not to acquire a transformation technique whose proper effectiveness would remain to be demonstrated. The economic model finally deserves a look, without lapsing into named accusation: certain premium personal-development offers borrow from network marketing, one climbs tiers, one becomes an ambassador, one resells the acquired status, one is encouraged to recruit, so that the prosperity of the system depends on the flow of newcomers at least as much as on the effectiveness of the content for those already in it. One becomes certified to run sessions in turn, one takes a cut on the places one sells, one is invited to bring in one’s circle: the vocabulary is that of transformation, the mechanics that of a distribution network. This structure proves no deception by itself, but it aligns the incentives on the volume of sales, which invites examining the promise more closely rather than taking it at face value.
5. Success would prove effectiveness
There remains the knockout argument: success. If the figure is rich and followed by millions of people, then surely the method works, one tells oneself; so many people cannot be wrong, such a success cannot rest on nothing. Yet the number of followers and the fortune accumulated say nothing about the effectiveness of the method, no more than the popularity of a remedy ever established that it cured. From patent remedies of the nineteenth century, sold by the wagonload and defended by grateful customers, to the miracle diets that return every decade, the list of commercial successes with no demonstrated effect is long, and the gratitude of their devotees never changed anything about it. The history of crazes is full of products adored by crowds and devoid of proper effect: fervour measures the appeal, not the truth of what one adores. The appeal to wealth, for its part, confuses the commercial success of the seller with the effectiveness of the product sold, when one can grow rich selling what does not keep its promises, as one can fail with an excellent product poorly marketed.
What success really measures is something other than effectiveness. That a figure is rich and followed by millions of people proves that she sells well, which is a real skill, that of marketing, the stage and distribution, distinct from the effectiveness of what she sells. To fill a stadium, to hold a hall spellbound for twelve hours, to make thousands of people sign in one weekend are genuine know-how, rare and precious, which say how much the person sells, never whether what she sells produces the promised effect. Her fortune comes from an empire built on volume, sold-out events, licences, books, derivative products, whose revenues depend on the number of entries sold, not on a follow-up of the results obtained by the participants. Not a single line of this turnover is indexed on the number of lives really changed: one pays to enter, never by result, and the empire can prosper whether the promise holds or not. And the measurement, precisely, is lacking: there are hardly any trials comparing the proper effect of these methods with cheap alternatives, so that commercial success gives no information on the value taught (Theeboom et al. 2014). One would like to cite a study that followed over a year participants drawn at random against a comparable group kept apart; it is practically missing, and its absence leaves the question of proper effectiveness wholly open (Theeboom et al. 2014). Two very different talents are thus gathered in the great figures of the sector, a real talent for communication and selling, and a promise of personal effectiveness; the first is attested by commercial success, the second remains to be established, and nothing obliges one to credit the seller with the second on the strength of the first.
6. Honest concession and conclusion
It would be dishonest to end on an indictment, for real effects exist. Coaching produces documented effects, reframing, motivation, accountability, a feeling of belonging, but modest ones, and neither specific to a given figure nor proper to a high price, as the available meta-analyses show; some people come out of it genuinely more motivated, clearer on their priorities, better surrounded, and this gain is authentic even if it remains modest and not specific to the premium offer (Theeboom et al. 2014; Jones et al. 2015). It is explained in large part by the common factors, expectation, alliance, mobilisation, rather than by an ingredient proper to a premium method, which explains why a simple alternative can produce comparable results (Wampold 2015).
Skepticism must nonetheless remain symmetric, without stopping at the camp that suits. The evidence base on coaching remains limited and heterogeneous, sometimes tainted by conflicts of interest, which requires not overvaluing these meta-analyses, in one direction as in the other (Theeboom et al. 2014). And when a durable change occurs after a seminar, the merit goes first to the person who brought it about: the method may have played a role of catalyst or occasion, which is neither negligible nor the proof of an effectiveness that would be proper to it.
From all this emerges a portable tool, applicable to any premium offer. It consists in telling apart what wins adherence, the price, the crowd, the testimonial, the success, from what would establish effectiveness, a comparative proof, and it comes down to one question to ask oneself before any costly promise: what, here, would prove that this works better than an alternative ten times cheaper? If the answer is the price, the hall, the testimonials or the seller’s success, one is still on the side of what convinces, without having touched what establishes effectiveness. The question does not devalue the lived experience, it puts it back in its place, on the side of feeling. What would settle it is a trial comparing the method with a cheap alternative, at equal motivation; as long as it is lacking, the doubt about proper effectiveness remains justified, without one having to cry imposture for all that. The conclusion is less severe than it looks: it invites neither to flee these offers nor to rush into them, rather to know what one is buying, an experience sometimes strong and useful, of which nothing guarantees that it is worth its price in proper effectiveness as long as no fair comparison has shown it. It is the same caution as before any premium offer: to ask oneself, before paying, what would distinguish a real effectiveness from a fine staging, and to wait, before declaring oneself convinced, for the answer to stand on its own two feet.
