A cool concept by the designer Yairko

For the design of Keeples.com, we used 99designs. We learned a lot about how to run an effective contest, and we’d like to share some of those insights with you here, so you can run a great contest too.

Before Launch

1. Espionage
   Look at others’ successful contests, and try to see what makes them successful. See the price range, the brief, and the feedback quality.
2. Pick an appropriate price
   We went with $600 for the designer (which meant a cost of $699 for us). We were told to expect somewhere between 10-30 ideas, but we got an awesome 207 entries, which gave us the chance to look at a lot of great ideas. I’d suggest picking something in that range for websites since it seems to be a decent amount of money to get great designs, but not too ridiculous so you won’t go broke (especially if you have a budget like Keeples’ budget).
3. Know thy product
   It’s important to have a good idea of what components to have in the final work. You don’t need to know what it’ll look like (if you did, you wouldn’t need a designer!), but you do need to know what things it should have. I would suggest making three lists – things you definitely want, things you want but can compromise on, and things you definitely don’t want.
4. Be concise and detailed at the same time
   The idea is to be stingy with your words so the designers don’t have to spend hours going through the brief. But at the same time, you need to include as much as you can about what you want the design to look like. Tell them those things you want, maybe want, and definitely don’t want, and only include relevant information for those parts. Verbiage is the enemy.
   Stated another way, clarity is everything. Designers will find a million different creative ways to misinterpret what you’re saying, so you want to be clear the first time. Also, keep in mind that most designers are not native English speakers when writing the brief.
5. Examples
   In the design brief, include examples of sites you like and those you don’t. Make sure there’s some sort of consistency there, otherwise it’ll just be confusing.
6. Include relevant files
   Logos, rough sketches (VERY useful), any sort of pictures that demonstrate your ideas. You can make it very clear the final product shouldn’t look like that (or should, if that’s the case), but if you have an image that demonstrates functionality, definitely include it since it gives a clear idea of what you need.
7. Hidden Contests?
   We didn’t go for a hidden contest, and honestly, I couldn’t give you a fair opinion on that. I really don’t know whether it works, but an un-hidden contest worked great for us.
8. No Fries, Please
   Forgo the extras! They cost too much and yield too little. Totally not worth it.

After Launch

1. Continue the espionage
   Look at other contests and see the designers whose work you like. Take note of a) The entry number in the contest you like and b) What aspects of the designs you like.
2. Appreciate people
   Message those designers and tell them you liked their entry #x in the contest for “that dog floss website.” This is the BEST way to get designers you like to participate. Since you’ve already liked their work before, there’s a good chance you’ll like their work on your contest. And they like to be appreciated – who doesn’t? I would suggest messaging 20-40 people in this way. It’s time consuming, but SO worth it. You get great value.
3. Don’t Sleep
   You need to give tons of feedback, and the quicker the better. Here’s the pattern I followed: a) Give a star rating. b) Write feedback – first what you liked about the design c) Then what you want improved. And d) Specific instructions on how to get the next higher star rating. Part d is very important, don’t skip. This is really best when it’s done immediately after a design is submitted – luckily for us I was halfway around the world from Utsab when we ran the contest so one of us was awake at all times. That may not be the case for you, but I’d suggest checking your contest the last thing you do at night and the first thing in the morning, and at regular intervals during the day.
4. Don’t be discouraged
   The first versions of the design will look terrible. You can’t do anything about that. The first versions of Keeples were not at all what we wanted. But don’t lose faith – the designers are awesome and they’ll get it right if you keep giving them feedback. Tell them what you like and what you don’t like, and have faith.
5. Talk to the designers
   Some designers like to chat with you on gchat or skype – and that’s awesome. You should encourage this – you can get a lot done by talking to them while they’re on photoshop since they’ll do exactly what you ask and show you many iterations.
6. Deal with problems quickly
   It seems like allegations of copying are pretty common, and you should address them as soon as possible. If you’re unsure whether the allegation is true or not, you should ask 99designs. If not, then tell them that’s what you think. Some designers go as far as to give fake threats about reporting you – those are just ridiculous pressure tactics and you should ignore them. Once again, ask the 99designs staff if unsure. They’re pretty helpful.
7. Extend only as much as needed
   We extended ours by 5 days, which was a bit of a mistake. Giving so much feedback became extremely exhausting after a while, and we had a lot of trouble keeping up. Also, the flow of new ideas kind of slowed down. Extending was definitely appropriate to do, but 5 days was overkill.
8. Connect with talent
   One of the best things we got out of our contest is the contact info for three spectacular designers. We’ve gotten more work from them since, and we know they can be relied upon to produce awesome stuff.

In addition to this, we suggest reading the suggestions 99designs gives as well, there’s definitely useful stuff there.

Are there any other suggestions you’d give? What’s your experience running 99designs (or any other crowdsourcing site) contests? Tell us in the comments!

I apply this philosophy for both 10Pens and Keeples. Instead of naively hoping that the projects will succeed the first time, I ASSUME the first try isn’t going to be quite what I expect. Instead of hoping for success, I hope for feedback — so that we can learn from our first experiences and try again. It is even on our product roadmap to have several iterations of the projects. Basically, our plan is to have one iteration of Keeples development, where we actually build the website. Then we market it. While we are marketing Keeples, we have one iteration of developing 10Pens. Then, while we are marketing 10Pens, we go back to the drawing board for Keeples and try again using the feedback we got during our marketing phase. That is, we build one website, then test it out on a population. Then we build it again, and repeat. I plan on putting at least four solid attempts into each project, 10Pens and Keeples, before I even think of calling it quits.

You could argue that you may not have the time nor the resources to try four times on a project. However, I would argue that if you are not capable of trying again after you fail, then you probably should not be launching a startup in the first place.

Thus, I decided to make a new marketing effort for Keeples! Today, I dressed up, gathered my laptop and a bag of candy, and drove over to a gas station that my parents manage. I set up shop next to the cashier and fastened a sign to the counter with some scotch-tape: “FREE CANDY FOR USER STUDY”.

The Results:

The results were quite positive. During the two hours I was there, I asked eleven people to participate, and six of them agreed. The feedback I got from the users was pretty informative. For example, I asked a couple of people for their opinions about our weekly email system:

Me: “Suppose we picked out 5 questions for you each week to answer, based on your expertise and interests. Would you think of the weekly email as a nuisance? Or would you actually answer the questions?”
User: “If I regularly used the site, then of course I wouldn’t mind.”

Another person from the study remarked that he would only answer questions which came directly from his friends. Strangers, he did not care for. Our user participation model, by the way, is a huge topic of interest for us, so we would appreciate any and all feedback from our readers about it.

The Challenges:

There were two main challenges I faced while performing the study:

1) Customer demographic. A good portion of the people coming in to the station did not speak much English and used the internet rarely if at all. Many people did not even have an email address. I could only ask a handful of people to participate, but fortunately the few people who I did ask were very helpful.

2) Question Quality. Many people were having difficulty coming up with a question on the spot. A couple of them did ask good specific questions which require human knowledge, such as “Where can I learn Bollywood dancing in San Jose, CA?”. However, most of the users asked really generic questions such as, “What is the capital of Greenland?” Keeples is certainly useful for the former question, because we will send out that question to everyone in our community who lives in San Jose and is interested in dance. However, you don’t need to use Keeples to figure out the capital of Greenland…
To solve this problem in the future, I will try to find ways to encourage users to ask high-quality questions.

What I have learned:

1) The sign helps a lot! It seems to make your user study look much more professional. I tried doing the same user study yesterday without a sign, and users were significantly less receptive.
2) Start out be telling them what they get out of it. “Hey, we’re giving out free candy today … if you help us out with a user study.”
3) Explicitly tell them how long it is going to take: “It will only take 30 seconds to a minute”. It does sound obvious, but it’s easy to forget how important that little piece of information is from the users’ point of view. I got this advice directly from one of the users in our study, who happened to be a researcher himself.
4) Have a visible basket of candy which you can point to.
5) A user was much more likely to agree to the user study if he saw someone else agree to it. Three of the user studies happened back-to-back-to-back. One person agreed; then the person standing directly behind him agreed; then the person standing directly behind HIM agreed. Nice.

The key here seems to be to give people something to LOOK at instead of simply verbally telling them something. Also, reassuring them that it is a low commitment on their part and reminding them that there is a juicy candy at the end of the study seems to help. Quick, simple, and clear was the goal. In fact, during the user study, I gained a new appreciation for all the effort we had put in to making the website as simple as possible to use. The fact that there was no registration/signup required for Keeples was absolutely essential to getting these users to participate, as many of them were rather impatient about the time investment.

Overall, the user studies seem to be quite effective. Tomorrow I will go back to the station, this time for six hours. Hopefully, I can recruit another 15 unique questions that way. I’ll post the results here after I have performed a few more user studies — and burned through a few more boxes of candy.

We had your ails in mind when we thought up Keeples. So the next time you want to know “What is SAS70,” like a good friend, Keeples will be there for you. When you need to feel special, answer some questions on Keeples, and we’ll put you on the MIP leaderboard. And always remember,

Keeples Loves You.

Finally, you should mark “tell everyone I know” on your calendar. This way, Keeples can be a good excuse for you to email people you haven’t talked to in a bit. Tell them your friends started the site and you wanted to let them know.  Yes, we are your friends.  We love to hear what you think about the website, and we want to know how we can make it better for you.

So, please explore the site, ask some questions, leave some feedback — oh, and tell everyone!

You can either ask questions by going to www.keeples.com or by emailing your question in the body of an email to ask@keeples.

This blog is our Keeples development and product blog. Currently, we’ve put up an initial version of Keeples with all the core functionality. This is our prototype/initial beta, so we would absolutely love any feedback you can give. Please let us know any and all thoughts you have about Keeples and any way you think it can be improved.