How to ensure high-quality talks for your conference

The goal of any conference program committee is to offer the most valuable content for their audience. While reviewing talk proposals is the committee's primary activity, this alone won't ensure that the program will provide maximum value.

Approaches that don't work

Over the years, various conference program committees have developed their own methods for ensuring program quality. For instance, some require proposers to include a video of past speaking engagements so members can see firsthand how effectively the speaker presents material. Unfortunately, this approach not only requires a significant time investment for committee members, but it also excludes new speakers who may be unable to provide a video.

Other conference committees request outlines of proposed talks or even drafts of presentation slides before a proposal is even accepted. This approach can alienate potential presenters by requiring setting the barrier to entry quite high. It also encourages rushed and inaccurate information, added hastily to meet submission requirements and not reflecting the real content of the talk. It's inadvisable for a program committee to judge proposal value based on this potentially unreliable information.

Take ownership of program quality

Both of these approaches shift the burden of program quality entirely onto proposers rather than keeping it with those who have the most control: The program committee. The key to receiving better proposals and talks is for the program committee to help people learn how to create better proposals and talks.

The program committee must take ownership of the quality of the event program. To help with this, program committees must actively assist proposers and speakers. Here's how to help shape a conference program that delivers maximum value.

Communicate expectations

The first step is to provide potential speakers with the information they'll need to create those proposals. If you don't set up appropriate expectations, you shouldn't be disappointed when people fail to meet them. Here are some of the expectations you should share:

• Types and lengths of presentations. If your conference includes workshops or tutorials, be very clear about the expected time commitment. For some events, a workshop is 90 minutes, while for others it can be three hours or more. Such terms aren't standardized, so don't assume proposers will know how you are using them.
• Topics the conference does—or does not—want on the program. Just because your event is about JavaScript, for example, does not mean you want to include talks about all possible JS subjects. Are you focusing on server-side JavaScript? A specific framework? Maybe the conference concentrates on JavaScript for the internet of things. Clearly communicate topic expectations to the proposers.
• Talk requirements. Some events have special requirements, such as requiring that all code and materials be available under a free or open source license by the time of the presentation. If your event has such requirements, tell proposers in advance so they can plan accordingly.
• Financial and travel support. Writing, traveling, and presenting require an investment of time and money. Tell potential speakers whether your conference can offset those expenses or not, so they can self-select out of events they can't afford to attend.
• Number of proposals to present. Many (especially newer) speakers don't realize that they can—and usually should—propose more than one talk for an event. Experienced speakers know how to play the CFP game and often propose many talks. If the program committee prefers to limit the number of talks by any single presenter, make this clear on the CFP form.

Share an audience profile

Every conference should communicate details about the audience. What sort of people will speakers be addressing? What is the audience's experience level and typical job role? If there are multiple tracks, it's likely the audience will differ for each one. Take a few moments to detail what proposers should expect for each track. Letting the proposers know this information in advance allows them to submit proposals that are suitable for that specific audience.

Be clear about the dates

A final set of expectations are the dates for the process. Some of the dates you should consider sharing on the CFP form include:

• Dates for the conference itself (clarifying which days are tutorials or hack days, if applicable)
• The start date for the CFP
• The end date for the CFP
• The anticipated date people should hear back about their proposals
• The anticipated date when the schedule will be available
• The anticipated date when conference registration will open

Naturally, these dates may need to change throughout the process. For instance, if the conference receives many more proposals than it expected, it may need to delay the response date for proposals. It's fine to change dates, as long as you communicate the changes via blog post, social media, or email, but sharing dates and deadlines will establish expectations for potential presenters.

Mentor and provide feedback on proposals

Once you've established expectations, the next step is to help presenters with their talk proposals. Even experienced speakers may not fully understand how to create a good proposal. They may have stumbled on something that worked once and have been cargo culting it ever since. New speakers may be entirely lost and not know how to construct a good proposal at all. You can address both of these problems by offering to mentor potential speakers.

Mentoring doesn't need to become complicated. I've found that using CFP office hours and email feedback work and scale well.

1. CFP office hours: Office hours are pre-declared times when potential speakers can find proposal help in the event chat system. This method is perfect for providing immediate feedback to potential speakers, especially as the CFP closing time looms. Chat-based office hours also have the advantage of allowing many people (on each side) to participate at once. While this could get chaotic, in practice it usually works well. It also allows potential speakers to help each other and learn from the feedback given to others.

2. Email feedback: Providing email feedback is great for program committees and potential speakers who need the flexibility of a more asynchronous approach. It supplies feedback for people whose schedules don't allow them to attend office hours in the chat system. It's also a good option for potential speakers who don't feel comfortable sharing their ideas in the public chat system.

Because each method has its advantages, consider using both for your event. Deb Nicholson, a founder and organizer of SeaGL, says the event has used both methods since 2015, and each year potential speakers tell them that the support they receive boosts their confidence in proposing talks to that and other events. By helping potential speakers with their proposals, SeaGL contributes to the rising tide that floats all conference boats.

However your event chooses to offer feedback, make sure that your mentors are familiar with how to write a good conference talk proposal and that they fully understand what your conference is looking for. If you don't have well-informed mentors, you may do more harm than good.

Keep the audience's perspective front and center

Once you've helped potential presenters craft their proposals, it's time to start reviewing them. Before you start, make sure that all reviewers are on the same page. Just as with potential speakers, you must define and communicate expectations for the proposal reviewers. Consider communicating the following information:

• How the scoring works and what each score means
• Whether reviewers should (or are required to) review proposals "blind" (without seeing who proposed the talk)
• What the program committee is looking for in that year's conference talks
• Audience profile
• Criteria that qualify a talk for a down vote
• Criteria that qualify a talk for an up vote

Reviewers should try to evaluate talks from the perspective of an audience member. What information will an audience member have to determine whether a talk is worth their time? Most attendees have no more than the abstract and the title. On the first pass, reviewers should try to rate talk proposals using only the information an audience member will have. Answering the question: "If I were an audience member, what value would I get out of this?" helps reviewers relate to, and maintain empathy with, the audience.

Focusing on audience needs also helps to reduce personal bias or favoritism. When you're thinking of the perspectives of others, it's a lot harder to be influenced your own. Naturally, reviewers should rely on their own personal knowledge—such as whether the subject is popular right now or (if proposals are not being reviewed blind) whether the proposer is a missing stair—but they should also be careful not to put their own needs or those of their employer before those of the audience.

Continue mentoring after talk acceptance

OK, so you've set up expectations, mentored presenters to craft a strong proposal, and done a great job selecting proposals, so you're ensured a great event schedule, right? Unfortunately, no. While you are likely to have a great schedule, the best way to be entirely sure is to continue the mentoring beyond proposals and on into writing and presenting a good talk.

Everyone needs an editor, whether they realize it or not. Even experienced speakers can use an extra pair of eyes on their material. Offer to help accepted speakers with their talks, and encourage less-experienced speakers to take advantage of the offer. Knowing that they have support can help alleviate some of the anxiety that new speakers may feel. Mentoring can also help ensure that the talk that's presented matches the abstract that the audience will use to choose which talk to see.

Yes, all of this mentoring can involve a lot of work. Keep in mind that only a small fraction of people will take you up on pre- and post-mentoring, which makes the work less than it actually sounds. It's worth it, though. We can't expect people to become better proposers and presenters if we aren't willing to put out a hand and lift them up to a higher level.