Ethics in Experimentation
When I joined my first startup in the late 1990’s, there was a term that was being thrown around in a negative manner… vaporware.
Vaporware described products and services that would get people excited, promising unrealistic expectations, yet never launched.
In extreme cases, people were sold products and scammed out of their money.
Things didn’t get a lot better in the early 2000’s either.
Companies would have a “code of ethics” handbook but their practices were still, well, rather questionable.
When I first stumbled upon the lean startup movement around 2010, some were already comparing the techniques to the vaporware tactics in the 1990’s.
At the time I didn’t view lean startup as such, but my opinions have always been ever evolving.
I’ve found in my work that most of the techniques we use can be boiled down into applying scientific method to business. The scientific approach we take in testing business ideas is very similar to research in the social sciences field.
Practitioners in social sciences already have five ethical principles to guide their work.
Be inclusive of different interests, values, funders, methods and perspectives.
Respect the privacy, autonomy, diversity, values, and dignity of individuals, groups and communities.
Actions are conducted with integrity throughout, employing the most appropriate methods for the research purpose.
Act with regard to social responsibilities in conducting and disseminating research.
Maximize benefit and minimize harm.
I attempted to reflect these while writing Testing Business Ideas, where I made some deliberate choices with experiments I included in the library, for example:
I omitted “imposter judo” or “impersonator” experiments. These stated that you should literally just copy your competitor’s product and rebrand it as your own for testing purposes. I have serious concerns about the ethics and legalities of doing so, besides I work with large companies that would be sued into oblivion. Instead, I took it upon myself to rebrand “boomerang” as competitive user testing. This means you can do competitive research without copying or plagiarizing another’s intellectual property.
I argued against fake testimonials when creating landing pages. It gives a false sense of social proof and influences new visitors to sign up. If you lack testimonials, then do the work of interviewing people that you’ve already provided similar value to and include their words, with permission. Don’t use services with stock images and fake names to give the illusion that you are further along in your development than you really are. Oh and here in the United States, 15 U.S. Code § 45 - Unfair methods of competition unlawful; prevention by Commission states that it is illegal.
I did not include the infamous “cancellable purchase order” experiment. This states that you should create a purchase order to determine real customer need, then after the customer signs it, just cancel and move on. Yay, demand validated! This is often characterized as “burning customers”. I can’t imagine the frustration this causes with potential customers, especially B2B ones who are in limited supply. The legality of cancellable purchase orders can be difficult to navigate depending on how it is written. Instead I suggest a Letter of Intent (LOI) as a non-legally binding one page agreement that closes the say / do gap.
I made it a point to write about diversity in teams. This is by far the topic that generates the most hate mail for me, but it is important for us to acknowledge the problems that arise from baking our biases right into the experiments and eventually into our products and businesses. Without diverse perspectives, it becomes too easy to overlook how your work may adversely impact underrepresented, overlooked and vulnerable communities.
One of the main reasons practitioners find all of this so difficult to navigate is that we are constantly dealing with extreme uncertainty when creating something new.
We may not fully understand the customer’s jobs, pains and gains.
We may not know our value proposition or even the solution we are about to build.
We don’t even know how we are going to make money.
This is why we should defer building as long as possible and test for desirability first before money exchanges hands. Even in a “presale” you need to adhere to federal securities laws that usually require you to deliver in 30 days. It doesn’t make sense to charge for something too early that you simply cannot deliver.
I feel much of my advice on ethics in experimentation can be distilled down into one guiding statement.
“Experiment with your customers, not on your customers.”
And this all starts with a simple, but needed conversation with your team.
When you are designing an experiment to de-risk your business idea, check in with your team and ask “Are we experimenting with our customers or on them?”
I fully realize it’s a balancing act between the need for ethically testing demand versus launching something that nobody cares about, but it is my hope that we can push these ideas forward without causing undue harm in the process.
Frequently Asked Questions
How can a team effectively balance the need for rapid innovation and ethical considerations, especially in highly competitive markets?
Balancing rapid innovation and ethical considerations is about integrating ethics into the innovation process itself. Teams should foster an environment where ethical deliberation is part of the ideation and development stages, ensuring that ethical considerations are not an afterthought but a fundamental aspect of competitive strategy.
Are there specific guidelines or frameworks for evaluating the ethical implications of business experiments in different industries or sectors?
Guidelines for ethical experimentation vary across industries, but the key is to develop a framework that considers the specific impacts of experiments in your sector. This involves understanding the unique ethical dimensions and potential risks associated with your industry's products, services, and customer interactions.
How can organizations ensure ongoing ethical compliance and awareness among team members as business strategies and market conditions evolve?
Ensuring ongoing ethical compliance in a changing business landscape requires continuous education and communication. Teams should be encouraged to engage in regular discussions about ethics, and leadership should model ethical behavior. Additionally, updating ethical guidelines to reflect new challenges or changes in the business environment is essential to maintain relevance and effectiveness.
